You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@vcl.apache.org by "Kumar, Amit H." <AH...@odu.edu> on 2010/04/01 21:25:12 UTC
VCL Getting Reservation to work
Hi All again,
After a long break I am back to where, I was unable to get the Reservation to work:
So, I was able to get Image capture work smoothly with the only short come of not completing the automatically created reservation process after image capture. Which failed at the point after registering the VM on the VMhost(ESXi). It does send the correct information and commands for registering the VM but it registeres the VM as "Unknown";
Following upon Ryan's comment on un mounting and remounting the NFS store. I started to make reservations and yet haven't been successful.
As usual something silly/unknown that I can't see.
Two unusual things:
1) esx.pm:node_status(814)|could not set ssh identity variable for image type= host=
2) "SSH exits with the exit status of the remote command or with 255 if an error occurred, output:" had same issue during image capture, and again Ryan pointed out a workaround to point to my NFS store directly. "Pushing the problem, doesn't not necessarily solve it" :)
Please advise.......................................
3 2010-04-01 11:04:58|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:04:58
4 2010-04-01 11:05:03|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:03
5 2010-04-01 11:05:08|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:08
6 2010-04-01 11:05:13|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:13
7 2010-04-01 11:05:18|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:18
8 2010-04-01 11:05:23|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:23
9 2010-04-01 11:05:28|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:28
10 2010-04-01 11:05:33|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:33
11 2010-04-01 11:05:38|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:38
12 2010-04-01 11:05:43|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:43
13 2010-04-01 11:05:48|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:48
14 2010-04-01 11:05:53|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:53
15 2010-04-01 11:05:53|5754|8:8|new|utils.pm:reservation_being_processed(9634)|computerloadlog 'begin' entry does NOT exist for reservation 8
16 2010-04-01 11:05:53|5754|8:8|new|utils.pm:reservation_being_processed(9686)|reservation is NOT currently being processed
17 2010-04-01 11:05:53|5754|8:8|new|vcld:main(276)|reservation 8 is NOT already being processed
18
19 |5754|8:8|new| ---- WARNING ----
20 |5754|8:8|new| 2010-04-01 11:05:53|5754|8:8|new|utils.pm:get_request_info(5218)|preferredimageid is not set for computer id=2
21 |5754|8:8|new| ( 0) utils.pm, notify (line: 737)
22 |5754|8:8|new| (-1) utils.pm, get_request_info (line: 5218)
23 |5754|8:8|new| (-2) vcld, main (line: 280)
24
25 2010-04-01 11:05:53|5754|8:8|new|utils.pm:get_request_info(5354)|standalone affiliation found: Local
26 2010-04-01 11:05:53|5754|8:8|new|utils.pm:get_management_node_info(6815)|management node info retrieved from database for colbert
27 2010-04-01 11:05:53|5754|8:8|new|vcld:main(281)|retrieved request information from database
28 2010-04-01 11:05:53|5754|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 8
29 2010-04-01 11:05:53|5754|8:8|new|utils.pm:update_request_state(2186)|request 8 state updated to: pending, laststate to: new
30 2010-04-01 11:05:53|5754|8:8|new|utils.pm:insertloadlog(4710)|inserted computer=2, begin, beginning to process, state is new
31 2010-04-01 11:05:53|5754|8:8|new|vcld:make_new_child(509)|loaded VCL::new module
32 2010-04-01 11:05:53|12084|8:8|new|vcld:make_new_child(582)|vcld environment variable set to 0 for this process
33 2010-04-01 11:05:53|12084|8:8|new|Module.pm:new(132)|constructor called, class=VCL::new
34 2010-04-01 11:05:53|5754|8:8|new|vcld:make_new_child(566)|current number of forked kids: 1
35 2010-04-01 11:05:53|12084|8:8|new|Module.pm:new(154)|VCL::new object created
36 2010-04-01 11:05:53|12084|8:8|new|State.pm:initialize(85)|obtained a database handle for this state process, stored as $ENV{dbh}
37 2010-04-01 11:05:53|12084|8:8|new|State.pm:check_image_os(820)|no corrections need to be made, not an imaging request, returning 1
38 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 8
39 2010-04-01 11:05:53|12084|8:8|new|utils.pm:rename_vcld_process(7863)|reservation count: 1
40 2010-04-01 11:05:53|12084|8:8|new|utils.pm:rename_vcld_process(7882)|PARENTIMAGE: 1
41 2010-04-01 11:05:53|12084|8:8|new|utils.pm:rename_vcld_process(7883)|SUBIMAGE: 0
42 2010-04-01 11:05:53|12084|8:8|new|utils.pm:rename_vcld_process(7901)|renamed process to 'vcld VCL::new 8:8 new'
43 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 8
44 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 8
45 2010-04-01 11:05:53|12084|8:8|new|State.pm:initialize(121)|attempting to load provisioning module: VCL::Module::Provisioning::esx
46 2010-04-01 11:05:53|12084|8:8|new|esx.pm:initialize(138)|esx vmware toolkit root path found: /usr/lib/vmware-vcli/apps
47 2010-04-01 11:05:53|12084|8:8|new|esx.pm:initialize(140)|vmware ESX module initialized
48 2010-04-01 11:05:53|12084|8:8|new|State.pm:initialize(127)|VCL::Module::Provisioning::esx module loaded
49 2010-04-01 11:05:53|12084|8:8|new|Module.pm:new(132)|constructor called, class=VCL::Module::Provisioning::esx
50 2010-04-01 11:05:53|12084|8:8|new|Module.pm:new(154)|VCL::Module::Provisioning::esx object created
51 2010-04-01 11:05:53|12084|8:8|new|esx.pm:initialize(138)|esx vmware toolkit root path found: /usr/lib/vmware-vcli/apps
52 2010-04-01 11:05:53|12084|8:8|new|esx.pm:initialize(140)|vmware ESX module initialized
53 2010-04-01 11:05:53|12084|8:8|new|State.pm:initialize(131)|VCL::Module::Provisioning::esx provisioner object created
54 2010-04-01 11:05:53|12084|8:8|new|State.pm:initialize(145)|attempting to load OS module: VCL::Module::OS::Linux
55 2010-04-01 11:05:53|12084|8:8|new|State.pm:initialize(151)|VCL::Module::OS::Linux module loaded
56 2010-04-01 11:05:53|12084|8:8|new|Module.pm:new(132)|constructor called, class=VCL::Module::OS::Linux
57 2010-04-01 11:05:53|12084|8:8|new|Module.pm:new(154)|VCL::Module::OS::Linux object created
58 2010-04-01 11:05:53|12084|8:8|new|State.pm:initialize(154)|VCL::Module::OS::Linux OS object created
59 2010-04-01 11:05:53|12084|8:8|new|State.pm:initialize(169)|returning 1
60 2010-04-01 11:05:53|12084|8:8|new|vcld:make_new_child(591)|VCL::new object created and initialized
61 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 8
62 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1946)|attempting to retrieve current state of computer esx35-hypervisor-1 from the database
63 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1977)|retrieved current state of computer esx35-hypervisor-1 from the database: available
64 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(697)|data structure updated: $self->request_data->{reservation}{8}{computer}{state}{name}
65 |12084|8:8|new| computer_state_name = available
66
67 |12084|8:8|new| ---- WARNING ----
68 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_id: $self->request_data->{reservation}{8}{computer}{preferredimage}{id}
69 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
70 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
71 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
72 |12084|8:8|new| (-3) new.pm, process (line: 106)
73 |12084|8:8|new| (-4) vcld, make_new_child (line: 594)
74 |12084|8:8|new| (-5) vcld, main (line: 341)
75
76
77 |12084|8:8|new| ---- WARNING ----
78 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_name: $self->request_data->{reservation}{8}{computer}{preferredimage}{name}
79 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
80 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
81 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
82 |12084|8:8|new| (-3) new.pm, process (line: 107)
83 |12084|8:8|new| (-4) vcld, make_new_child (line: 594)
84 |12084|8:8|new| (-5) vcld, main (line: 341)
85
86 2010-04-01 11:05:53|12084|8:8|new|utils.pm:get_management_node_id(9364)|environment variable: 1
87 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(134)|reservation is parent = 1
88 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(135)|preload only = 0
89 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(136)|originating request state = new
90 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(137)|originating request laststate = new
91 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(138)|originating computer state = available
92 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(139)|originating computer type = blade
93 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 8
94 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1946)|attempting to retrieve current state of computer esx35-hypervisor-1 from the database
95 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1977)|retrieved current state of computer esx35-hypervisor-1 from the database: available
96 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(697)|data structure updated: $self->request_data->{reservation}{8}{computer}{state}{name}
97 |12084|8:8|new| computer_state_name = available
98
99 |12084|8:8|new| ---- WARNING ----
100 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_id: $self->request_data->{reservation}{8}{computer}{preferredimage}{id}
101 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
102 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
103 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
104 |12084|8:8|new| (-3) new.pm, computer_not_being_used (line: 750)
105 |12084|8:8|new| (-4) new.pm, process (line: 170)
106 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
107 |12084|8:8|new| (-6) vcld, main (line: 341)
108
109
110 |12084|8:8|new| ---- WARNING ----
111 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_name: $self->request_data->{reservation}{8}{computer}{preferredimage}{name}
112 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
113 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
114 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
115 |12084|8:8|new| (-3) new.pm, computer_not_being_used (line: 751)
116 |12084|8:8|new| (-4) new.pm, process (line: 170)
117 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
118 |12084|8:8|new| (-6) vcld, main (line: 341)
119
120 2010-04-01 11:05:53|12084|8:8|new|utils.pm:get_management_node_id(9364)|environment variable: 1
121 2010-04-01 11:05:53|12084|8:8|new|new.pm:computer_not_being_used(788)|esx35-hypervisor-1 state is available
122 2010-04-01 11:05:53|12084|8:8|new|new.pm:computer_not_being_used(798)|esx35-hypervisor-1 is available, its state is available
123 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(170)|esx35-hypervisor-1 is not being used
124 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 8
125 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1946)|attempting to retrieve current state of computer esx35-hypervisor-1 from the database
126 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1977)|retrieved current state of computer esx35-hypervisor-1 from the database: available
127 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(697)|data structure updated: $self->request_data->{reservation}{8}{computer}{state}{name}
128 |12084|8:8|new| computer_state_name = available
129
130 |12084|8:8|new| ---- WARNING ----
131 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_id: $self->request_data->{reservation}{8}{computer}{preferredimage}{id}
132 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
133 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
134 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
135 |12084|8:8|new| (-3) new.pm, reload_image (line: 480)
136 |12084|8:8|new| (-4) new.pm, process (line: 266)
137 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
138 |12084|8:8|new| (-6) vcld, main (line: 341)
139
140
141 |12084|8:8|new| ---- WARNING ----
142 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_name: $self->request_data->{reservation}{8}{computer}{preferredimage}{name}
143 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
144 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
145 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
146 |12084|8:8|new| (-3) new.pm, reload_image (line: 481)
147 |12084|8:8|new| (-4) new.pm, process (line: 266)
148 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
149 |12084|8:8|new| (-6) vcld, main (line: 341)
150
151 2010-04-01 11:05:53|12084|8:8|new|utils.pm:get_management_node_id(9364)|environment variable: 1
152 2010-04-01 11:05:53|12084|8:8|new|new.pm:reload_image(514)|calling VCL::Module::Provisioning::esx->node_status()
153 2010-04-01 11:05:53|12084|8:8|new|utils.pm:insertloadlog(4710)|inserted computer=2, statuscheck, checking status of node
154
155 |12084|8:8|new| ---- WARNING ----
156 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_vmhost_profile_vmpath: $self->request_data->{reservation}{8}{computer}{vmhost}{vmprofile}{vmpath}
157 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
158 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
159 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
160 |12084|8:8|new| (-3) esx.pm, node_status (line: 776)
161 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
162 |12084|8:8|new| (-5) new.pm, process (line: 266)
163 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
164
165
166 |12084|8:8|new| ---- WARNING ----
167 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_vmhost_profile_datastore_path: $self->request_data->{reservation}{8}{computer}{vmhost}{vmprofile}{datastorepath}
168 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
169 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
170 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
171 |12084|8:8|new| (-3) esx.pm, node_status (line: 777)
172 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
173 |12084|8:8|new| (-5) new.pm, process (line: 266)
174 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
175
176
177 |12084|8:8|new| ---- WARNING ----
178 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_vmhost_type: $self->request_data->{reservation}{8}{computer}{vmhost}{type}
179 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
180 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
181 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
182 |12084|8:8|new| (-3) esx.pm, node_status (line: 779)
183 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
184 |12084|8:8|new| (-5) new.pm, process (line: 266)
185 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
186
187
188 |12084|8:8|new| ---- WARNING ----
189 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_vmhost_hostname: $self->request_data->{reservation}{8}{computer}{vmhost}{hostname}
190 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
191 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
192 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
193 |12084|8:8|new| (-3) esx.pm, node_status (line: 780)
194 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
195 |12084|8:8|new| (-5) new.pm, process (line: 266)
196 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
197
198
199 |12084|8:8|new| ---- WARNING ----
200 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_vmhost_image_name: $self->request_data->{reservation}{8}{computer}{vmhost}{imagename}
201 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
202 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
203 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
204 |12084|8:8|new| (-3) esx.pm, node_status (line: 781)
205 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
206 |12084|8:8|new| (-5) new.pm, process (line: 266)
207 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
208
216 2010-04-01 11:05:53|12084|8:8|new|esx.pm:node_status(787)|Entering node_status, checking status of esx35-hypervisor-1
217 2010-04-01 11:05:53|12084|8:8|new|esx.pm:node_status(788)|request_for_imaging: 0
218 2010-04-01 11:05:53|12084|8:8|new|esx.pm:node_status(789)|requeseted image name: esx3-basewinxp-v0
219 Use of uninitialized value in string eq at
220 /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 804 (#1)
221 (W uninitialized) An undefined value was used as if it were already
222 defined. It was interpreted as a "" or a 0, but maybe it was a mistake.
223 To suppress this warning assign a defined value to your variables.
224
225 To help you figure out what was undefined, perl tells you what operation
226 you used the undefined value in. Note, however, that perl optimizes your
227 program and the operation displayed in the warning may not necessarily
228 appear literally in your program. For example, "that $foo" is
229 usually optimized into "that " . $foo, and the warning will refer to
230 the concatenation (.) operator, even though there is no . in your
231 program.
232
233
234 |12084|8:8|new| ---- WARNING ----
235 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|vcld:warning_handler(636)|Use of uninitialized value in string eq at /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 804.
236 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
237 |12084|8:8|new| (-1) vcld, warning_handler (line: 636)
238 |12084|8:8|new| (-2) esx.pm, node_status (line: 804)
239 |12084|8:8|new| (-3) new.pm, reload_image (line: 518)
240 |12084|8:8|new| (-4) new.pm, process (line: 266)
241 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
242 |12084|8:8|new| (-6) vcld, main (line: 341)
243
244
245 |12084|8:8|new| ---- WARNING ----
246 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|vcld:warning_handler(636)|Use of uninitialized value in pattern match (m//) at /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 811.
247 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
248 |12084|8:8|new| (-1) vcld, warning_handler (line: 636)
249 |12084|8:8|new| (-2) esx.pm, node_status (line: 811)
250 |12084|8:8|new| (-3) new.pm, reload_image (line: 518)
251 |12084|8:8|new| (-4) new.pm, process (line: 266)
252 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
253 |12084|8:8|new| (-6) vcld, main (line: 341)
254
255
256 |12084|8:8|new| ---- WARNING ----
257 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|vcld:warning_handler(636)|Use of uninitialized value in concatenation (.) or string at /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 815.
258 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
259 |12084|8:8|new| (-1) vcld, warning_handler (line: 636)
260 |12084|8:8|new| (-2) esx.pm, node_status (line: 814)
261 |12084|8:8|new| (-3) new.pm, reload_image (line: 518)
262 |12084|8:8|new| (-4) new.pm, process (line: 266)
263 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
264 |12084|8:8|new| (-6) vcld, main (line: 341)
265
266
267 |12084|8:8|new| ---- WARNING ----
268 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|vcld:warning_handler(636)|Use of uninitialized value in concatenation (.) or string at /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 815.
269 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
270 |12084|8:8|new| (-1) vcld, warning_handler (line: 636)
271 |12084|8:8|new| (-2) esx.pm, node_status (line: 814)
272 |12084|8:8|new| (-3) new.pm, reload_image (line: 518)
273 |12084|8:8|new| (-4) new.pm, process (line: 266)
274 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
275 |12084|8:8|new| (-6) vcld, main (line: 341)
276
277
278 |12084|8:8|new| ---- WARNING ----
279 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|vcld:warning_handler(636)|Use of uninitialized value in concatenation (.) or string at /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 815.
280 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
281 |12084|8:8|new| (-1) vcld, warning_handler (line: 636)
282 |12084|8:8|new| (-2) esx.pm, node_status (line: 814)
283 |12084|8:8|new| (-3) new.pm, reload_image (line: 518)
284 |12084|8:8|new| (-4) new.pm, process (line: 266)
285 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
286 |12084|8:8|new| (-6) vcld, main (line: 341)
287
288 No recipient addresses found in header
289 2010-04-01 11:05:54|12084|8:8|new|utils.pm:mail(1348)|SUCCESS -- Sending mail To: , PROBLEM -- esx.pm
290
291 |12084|8:8|new| ---- CRITICAL ----
292 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|esx.pm:node_status(814)|could not set ssh identity variable for image type= host=
293 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
294 |12084|8:8|new| (-1) esx.pm, node_status (line: 814)
295 |12084|8:8|new| (-2) new.pm, reload_image (line: 518)
296 |12084|8:8|new| (-3) new.pm, process (line: 266)
297 |12084|8:8|new| (-4) vcld, make_new_child (line: 594)
298 |12084|8:8|new| (-5) vcld, main (line: 341)
299
300 2010-04-01 11:05:54|12084|8:8|new|esx.pm:node_status(819)|checking if esx35-hypervisor-1 is pingable
301 2010-04-01 11:05:54|12084|8:8|new|esx.pm:node_status(822)|esx35-hypervisor-1 is pingable (1)
302 2010-04-01 11:05:54|12084|8:8|new|esx.pm:node_status(836)|Trying to ssh...
303 2010-04-01 11:05:54|12084|8:8|new|utils.pm:run_ssh_command(6180)|executing SSH command on esx35-hypervisor-1:
304 |12084|8:8|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'uname -s' 2>&1
305
306 |12084|8:8|new| ---- WARNING ----
307 |12084|8:8|new| 2010-04-01 11:05:54|12084|8:8|new|utils.pm:run_ssh_command(6252)|attempt 1/3: failed to execute SSH command on esx35-hypervisor-1: uname -s, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
308 |12084|8:8|new| ssh output (uname -s): VMkernel
309 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
310 |12084|8:8|new| (-1) utils.pm, run_ssh_command (line: 6252)
311 |12084|8:8|new| (-2) utils.pm, _sshd_status (line: 2755)
312 |12084|8:8|new| (-3) esx.pm, node_status (line: 839)
313 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
314 |12084|8:8|new| (-5) new.pm, process (line: 266)
315 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
316
317 2010-04-01 11:05:54|12084|8:8|new|utils.pm:run_ssh_command(6172)|sleeping for 2 seconds before making next SSH attempt
318 2010-04-01 11:05:56|12084|8:8|new|utils.pm:run_ssh_command(6184)|attempt 2/3: executing SSH command on esx35-hypervisor-1:
319 |12084|8:8|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'uname -s' 2>&1
320
321 |12084|8:8|new| ---- WARNING ----
322 |12084|8:8|new| 2010-04-01 11:05:56|12084|8:8|new|utils.pm:run_ssh_command(6252)|attempt 2/3: failed to execute SSH command on esx35-hypervisor-1: uname -s, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
323 |12084|8:8|new| ssh output (uname -s): VMkernel
324 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
325 |12084|8:8|new| (-1) utils.pm, run_ssh_command (line: 6252)
326 |12084|8:8|new| (-2) utils.pm, _sshd_status (line: 2755)
327 |12084|8:8|new| (-3) esx.pm, node_status (line: 839)
328 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
329 |12084|8:8|new| (-5) new.pm, process (line: 266)
330 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
331
332 2010-04-01 11:05:56|12084|8:8|new|utils.pm:run_ssh_command(6172)|sleeping for 15 seconds before making next SSH attempt
333 2010-04-01 11:05:58|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:05:58
334 2010-04-01 11:06:03|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:06:03
335 2010-04-01 11:06:08|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 11:06:08
336
Thank you,
Amit
Re: VCL Getting Reservation to work
Posted by Andy Kurth <an...@ncsu.edu>.
The screen shot attachments weren't included. I don't this this list allows
some files to be attached. Can you please send the table contents using the
output from normal MySQL select statements:
SELECT * from image;
...
The output should be delimited so that it can easily be pasted into a spreadsheet.
-Andy
Kumar, Amit H. wrote:
> Attached are the screen shots.
>
>
> Thank you,
> Amit
>
>
>> -----Original Message-----
>> From: Ryan Johnson [mailto:rjohn06@gwmail.gwu.edu]
>> Sent: Thursday, April 01, 2010 4:23 PM
>> To: vcl-dev@incubator.apache.org
>> Subject: [Spam: 6.0] Re: [Spam: 6.0] Re: VCL Getting Reservation to work
>>
>> Yea, that is a little hard on the eyes, any chance you could send screen
>> shots in a view where the columns are lined up with the column names?
>>
>> On Thu, Apr 1, 2010 at 4:00 PM, Kumar, Amit H. <AH...@odu.edu> wrote:
>>
>>> The Savior! Thank you Ryan getting back!!!
>>>
>>> Not sure if this is the best format for readability...please let me know
>>> otherwise.
>>>
>>> I am also a recent log that I tried with verbose ssh enabled etc, I haven't
>>> looked at it yet. It went past my previous log. .....................
>>>
>>> Image: Table: Rows: 10
>>> ====================
>>> id name prettyname ownerid platformid OSid imagemetaid minram
>>> minprocnumber minprocspeed minnetwork maxconcurrent reloadtime deleted
>>> test lastupdate forcheckout maxinitialtime project size architecture
>>> description usage basedoffrevisionid
>>> 1 winxp-base1-v0 No Apps (WinXP) 1 [->admin] 1 [->i386] 7 [->winxp] NULL 0
>>> 1 0 10 NULL 14 0 0 2007-04-11 16:07:38 1 0 vcl 1045 x86 NULL NULL 0
>>> 2 rhel4-base2-v0 Red Hat Enterprise Linux 4.4 Base (KS) 1 [->admin] 1
>>> [->i386] 12 [->rhel4] NULL 1024 1 1024 100 NULL 13 0 0 2007-03-02 16:33:33 1
>>> 0 vcl 0 x86 NULL NULL 0
>>> 3 rh4image-base3-v0 RHEL4 base (image) 1 [->admin] 1 [->i386] 13
>>> [->rh4image] NULL 10 1 1024 100 NULL 10 0 0 2007-01-24 15:02:07 1 0 vcl 1450
>>> x86 NULL NULL 0
>>> 4 noimage No Image 1 [->admin] 1 [->i386] 2 [->sun4x_58] NULL 0 1 0 10 NULL
>>> 0 0 0 NULL 1 0 vcl 1450 x86 NULL NULL 0
>>> 5 rhfc5-fc5base5-v0 Red Hat Fedora Core 5 base (KS) 1 [->admin] 1 [->i386]
>>> 15 [->rhfc5] NULL 1024 1 1024 100 NULL 13 0 0 2006-10-02 10:04:24 1 0 vcl
>>> 1450 x86 NULL NULL 0
>>> 6 fc5image-FC5baseRHFC56-v0 FC5base(RHFC5) 1 [->admin] 1 [->i386] 14
>>> [->fc5image] NULL 64 1 500 10 NULL 10 0 0 2006-10-02 14:13:26 1 0 vcl 0 x86
>>> NULL NULL 0
>>> 7 vmwarewinxp-base7-v1 No Apps (WinXP vmware) 1 [->admin] 1 [->i386] 16
>>> [->vmwarewinxp] NULL 512 1 1024 100 NULL 5 0 0 2007-04-04 09:45:38 1 0 vcl
>>> 3244 x86 NULL NULL 0
>>> 8 rh4image-VMwareserverhostRHEL48-v3 VMware server host (RHEL4) 1 [->admin]
>>> 1 [->i386] 13 [->rh4image] NULL 64 1 500 10 NULL 10 0 0 2007-11-26 11:23:53
>>> 1 0 vcl 1450 x86 NULL NULL 0
>>> 9 esx35-base-v0 VMware ESX 3.5 standard server 1 [->admin] 1 [->i386] 20
>>> [->esx35] NULL 2048 2 2000 100 NULL 9 0 0 2008-03-24 14:23:54 1 0 vcl 1450
>>> x86 NULL NULL 0
>>> 10 esx3-basewinxp-v0 No Apps (ESXi WinXP) 1 [->admin] 1 [->i386] 20
>>> [->esx35] NULL 512 1 1024 100 NULL 9 0 0 2010-03-24 17:35:12 1 0 vcl 4100
>>> x86 NULL NULL 0
>>> /End Image Table====================
>>>
>>>
>>> ImageRevision Table: Rows: 11
>>> ===========================
>>> id imageid revision userid datecreated deleted datedeleted
>>> production comments imagename
>>> 1 1 [->No Apps (WinXP)] 0 1 [->admin] 2007-03-01 14:46:26 0 NULL 1 NULL
>>> winxp-base1-v0
>>> 2 2 [->Red Hat Enterprise Linux 4.4 Base (KS)] 0 1 [->admin] 2006-09-22
>>> 16:33:24 0 NULL 1 NULL rhel4-base2-v0
>>> 3 3 [->RHEL4 base (image)] 0 1 [->admin] 2007-01-24 15:04:11 0 NULL 1 NULL
>>> rh4image-base3-v0
>>> 4 4 [->No Image] 0 1 [->admin] 1980-01-01 00:00:00 0 NULL 1 NULL noimage
>>> 5 5 [->Red Hat Fedora Core 5 base (KS)] 0 1 [->admin] 2006-10-02 10:04:24 0
>>> NULL 1 NULL rhfc5-fc5base5-v0
>>> 6 6 [->FC5base(RHFC5)] 0 1 [->admin] 2006-10-02 14:13:26 0 NULL 1 NULL
>>> fc5image-FC5baseRHFC56-v0
>>> 7 7 [->No Apps (WinXP vmware)] 0 1 [->admin] 2007-04-03 11:24:12 0 NULL 1
>>> NULL vmwarewinxp-base7-v0
>>> 8 8 [->VMware server host (RHEL4)] 0 1 [->admin] 2007-01-30 10:50:56 0 NULL
>>> 1 NULL rh4image-VMwareserverhostRHEL48-v0
>>> 9 9 [->VMware ESX 3.5 standard server] 0 1 [->admin] 2008-03-24 14:23:54 0
>>> NULL 1 NULL esx35-base-v0
>>> 10 7 [->No Apps (WinXP vmware)] 1 1 [->admin] 2010-03-18 14:56:50 0 NULL 1
>>> NULL vmwarewinxp-base7-v1
>>> 11 10 [->No Apps (ESXi WinXP)] 0 1 [->admin] 2010-03-24 17:35:12 0 NULL 1
>>> NULL esx3-basewinxp-v0
>>> /End ImageRevision Table====================
>>>
>>>
>>> Computer Table: Rows: 3
>>> ============================================
>>> id stateid ownerid platformid scheduleid currentimageid
>>> preferredimageid nextimageid imagerevisionid RAM procnumber procspeed
>>> network hostname IPaddress privateIPaddress eth0macaddress
>>> eth1macaddress type provisioningid drivetype deleted notes lastcheck
>>> location dsa dsapub rsa rsapub host hostpub vmhostid vmtypeid
>>> 1 10 [->maintenance] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0
>>> 0 0 512 2 2200 1000 resxi-2.odu.edu 128.82.128.218 NULL NULL NULL
>>> virtualmachine 4 [->vmware_esx] hda 1 NULL NULL NULL NULL NULL NULL
>> NULL
>>> [BLOB - 0B] NULL NULL NULL
>>> 2 2 [->available] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0
>>> 0 20480 8 2200 1000 esx35-hypervisor-1.odu.edu 128.82.128.252 NULL NULL
>>> NULL blade 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL [BLOB
>> -
>>> 0B] NULL NULL NULL
>>> 3 5 [->failed] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0 0
>>> 512 2 2200 100 esx3-basewinxp-v0.odu.edu 128.82.128.217 NULL NULL NULL
>>> virtualmachine 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL
>> [BLOB
>>> - 0B] NULL 1 NULL
>>> /End Computer Table======================================
>>>
>>>
>>> Vmprofile Table: Rows: 4
>>> ========================
>>> id profilename vmtypeid imageid nasshare datastorepath vmpath
>>> virtualswitch0 virtualswitch1 vmdisk username password
>>> 1 VMware GSX standard 3 [->vmwareGSX] 8 [->VMware server host (RHEL4)] NULL
>>> /var/lib/vmware/Virtual Machines NULL VMnet0 VMnet2 localdisk NULL NULL
>>> 2 Vmware ESX standard network mounted share 5 [->vmwareESX3] 9 [->VMware
>>> ESX 3.5 standard server] NULL /vmfs/volumes/nfs1 /vmfs/volumes/storage1 VM
>>> Network Virtual Machine Public Network networkdisk NULL NULL
>>> 3 Vmware ESX standard localdisk 6 [->vmwareESXi] 9 [->VMware ESX 3.5
>>> standard server] NULL 192.168.129.37:/install/image/vmware_images
>>> 192.168.129.37:/install/image/vmware_images VM Network Private Vlan328
>>> localdisk root
>>> 4 Vmware ESX SAN 5 [->vmwareESX3] 9 [->VMware ESX 3.5 standard server]
>> NULL
>>> /vmfs/volumes/NetApp /vmfs/volumes/storage1 Intranet2 MCNC Public
>>> networkdisk NULL NULL
>>> /End Vmprofile===============================
>>>
>>> VMhost Table: Rows: 1
>>> =====================
>>> id computerid vmlimit vmprofileid vmkernalnic vmwaredisk
>>> 1 2 [->esx35-hypervisor-1.odu.edu] 2 3 NULL localdisk
>>> /End Vmhost Table==========================
>>>
>>>
>>> ================================= LOG FILE
>>> ===========================================
>>> 13 2010-04-01 15:37:19|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:37:19
>>> 14 2010-04-01 15:37:24|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:37:24
>>> 15 2010-04-01
>> 15:37:24|5754|9:9|deleted|utils.pm:reservation_being_processed(9634)|computerloadlog
>>> 'begin' entry does NOT exist for reservation 9
>>> 16 2010-04-01
>> 15:37:24|5754|9:9|deleted|utils.pm:reservation_being_processed(9686)|reservation
>>> is NOT currently being processed
>>> 17 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(276)|reservation 9 is
>>> NOT already being processed
>>> 18
>>> 19 |5754|9:9|deleted| ---- WARNING ----
>>> 20 |5754|9:9|deleted| 2010-04-01
>> 15:37:24|5754|9:9|deleted|utils.pm:get_request_info(5218)|preferredimageid
>>> is not set for computer id=2
>>> 21 |5754|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 22 |5754|9:9|deleted| (-1) utils.pm, get_request_info (line: 5218)
>>> 23 |5754|9:9|deleted| (-2) vcld, main (line: 280)
>>> 24
>>> 25 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:get_request_info(5354)|standalone
>>> affiliation found: Local
>>> 26 2010-04-01
>> 15:37:24|5754|9:9|deleted|utils.pm:get_management_node_info(6815)|management
>>> node info retrieved from database for colbert
>>> 27 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(281)|retrieved
>>> request information from database
>>> 28 2010-04-01
>>> 15:37:24|5754|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
>>> true: parent reservation ID for this request: 9
>>> 29 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:update_request_state(2186)|request
>>> 9 state updated to: pending, laststate to: deleted
>>> 30 2010-04-01
>> 15:37:24|5754|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
>>> entry not inserted, reservation has been deleted
>>> 31 2010-04-01
>>> 15:37:24|5754|9:9|deleted|vcld:make_new_child(489)|request will be processed
>>> by reclaim.pm
>>> 32 2010-04-01 15:37:24|5754|9:9|deleted|vcld:make_new_child(509)|loaded
>>> VCL::reclaim module
>>> 33 2010-04-01 15:37:24|12719|9:9|deleted|vcld:make_new_child(582)|vcld
>>> environment variable set to 0 for this process
>>> 34 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
>>> called, class=VCL::reclaim
>>> 35 2010-04-01
>>> 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::reclaim object created
>>> 36 2010-04-01
>>> 15:37:24|5754|9:9|deleted|vcld:make_new_child(566)|current number of forked
>>> kids: 1
>>> 37 2010-04-01
>>> 15:37:24|12719|9:9|deleted|State.pm:initialize(85)|obtained a database
>>> handle for this state process, stored as $ENV{dbh}
>>> 38 2010-04-01
>>> 15:37:24|12719|9:9|deleted|State.pm:check_image_os(820)|no corrections need
>>> to be made, not an imaging request, returning 1
>>> 39 2010-04-01
>>> 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
>>> true: parent reservation ID for this request: 9
>>> 40 2010-04-01
>> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7863)|reservation
>>> count: 1
>>> 41 2010-04-01
>> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7882)|PARENTIMAGE:
>>> 1
>>> 42 2010-04-01
>> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7883)|SUBIMAGE:
>>> 0
>>> 43 2010-04-01
>> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7901)|renamed
>>> process to 'vcld VCL::reclaim 9:9 deleted'
>>> 44 2010-04-01
>>> 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
>>> true: parent reservation ID for this request: 9
>>> 45 2010-04-01
>>> 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
>>> true: parent reservation ID for this request: 9
>>> 46 2010-04-01
>>> 15:37:24|12719|9:9|deleted|State.pm:initialize(121)|attempting to load
>>> provisioning module: VCL::Module::Provisioning::esx
>>> 47 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(138)|esx
>>> vmware toolkit root path found: /usr/lib/vmware-vcli/apps
>>> 48 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(140)|vmware
>>> ESX module initialized
>>> 49 2010-04-01
>>> 15:37:24|12719|9:9|deleted|State.pm:initialize(127)|VCL::Module::Provisioning::esx
>>> module loaded
>>> 50 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
>>> called, class=VCL::Module::Provisioning::esx
>>> 51 2010-04-01
>>> 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::Provisioning::esx
>>> object created
>>> 52 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(138)|esx
>>> vmware toolkit root path found: /usr/lib/vmware-vcli/apps
>>> 53 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(140)|vmware
>>> ESX module initialized
>>> 54 2010-04-01
>>> 15:37:24|12719|9:9|deleted|State.pm:initialize(131)|VCL::Module::Provisioning::esx
>>> provisioner object created
>>> 55 2010-04-01
>>> 15:37:24|12719|9:9|deleted|State.pm:initialize(145)|attempting to load OS
>>> module: VCL::Module::OS::Linux
>>> 56 2010-04-01
>>> 15:37:24|12719|9:9|deleted|State.pm:initialize(151)|VCL::Module::OS::Linux
>>> module loaded
>>> 57 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
>>> called, class=VCL::Module::OS::Linux
>>> 58 2010-04-01
>>> 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::OS::Linux object
>>> created
>>> 59 2010-04-01
>>> 15:37:24|12719|9:9|deleted|State.pm:initialize(154)|VCL::Module::OS::Linux
>>> OS object created
>>> 60 2010-04-01
>>> 15:37:24|12719|9:9|deleted|State.pm:initialize(169)|returning 1
>>> 61 2010-04-01
>>> 15:37:24|12719|9:9|deleted|vcld:make_new_child(591)|VCL::reclaim object
>>> created and initialized
>>> 62 2010-04-01
>>> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation
>>> remote IP is not defined
>>> 63 2010-04-01
>>> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1946)|attempting
>>> to retrieve current state of computer esx35-hypervisor-1 from the database
>>> 64 2010-04-01
>>> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1977)|retrieved
>>> current state of computer esx35-hypervisor-1 from the database: available
>>> 65 2010-04-01
>>> 15:37:24|12719|9:9|deleted|DataStructure.pm:_automethod(697)|data structure
>>> updated: $self->request_data->{reservation}{9}{computer}{state}{name}
>>> 66 |12719|9:9|deleted| computer_state_name = available
>>> 67 2010-04-01
>> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
>>> entry not inserted, reservation has been deleted
>>> 68 2010-04-01
>> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
>>> entry not inserted, reservation has been deleted
>>> 69 2010-04-01
>> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
>>> entry not inserted, reservation has been deleted
>>> 70 2010-04-01
>> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
>>> entry not inserted, reservation has been deleted
>>> 71 2010-04-01
>> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
>>> entry not inserted, reservation has been deleted
>>> 72 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(138)|computer
>>> type is blade
>>> 73 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(142)|request
>>> laststate is reserved, attempting to sanitize computer
>>> 74 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(172)|calling
>>> VCL::Module::OS::Linux::sanitize() subroutine
>>> 75 2010-04-01
>>> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation
>>> remote IP is not defined
>>> 76 2010-04-01
>> 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on esx35-hypervisor-1:
>>> 77 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'netstat -an' 2>&1
>>> 78
>>> 79 |12719|9:9|deleted| ---- WARNING ----
>>> 80 |12719|9:9|deleted| 2010-04-01
>> 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 1/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
>>> status: 255, SSH exits with the exit status of the remote command or with
>>> 255 if an error occurred, output:
>>> 81 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
>>> found
>>> 82 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 83 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 84 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
>>> 85 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
>>> 86 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 87 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 88 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 89
>>> 90 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 2 seconds before making next SSH attempt
>>> 91 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 2/3: executing SSH command on esx35-hypervisor-1:
>>> 92 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'netstat -an' 2>&1
>>> 93
>>> 94 |12719|9:9|deleted| ---- WARNING ----
>>> 95 |12719|9:9|deleted| 2010-04-01
>> 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 2/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
>>> status: 255, SSH exits with the exit status of the remote command or with
>>> 255 if an error occurred, output:
>>> 96 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
>>> found
>>> 97 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 98 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 99 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
>>> 100 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
>>> 101 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 102 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 103 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 104
>>> 105 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 15 seconds before making next SSH attempt
>>> 106 2010-04-01 15:37:29|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:37:29
>>> 107 2010-04-01 15:37:34|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:37:34
>>> 108 2010-04-01 15:37:39|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:37:39
>>> 109 2010-04-01
>> 15:37:39|5754|10:10|new|utils.pm:reservation_being_processed(9634)|computerloadlog
>>> 'begin' entry does NOT exist for reservation 10
>>> 110 2010-04-01
>> 15:37:39|5754|10:10|new|utils.pm:reservation_being_processed(9686)|reservation
>>> is NOT currently being processed
>>> 111 2010-04-01 15:37:39|5754|10:10|new|vcld:main(276)|reservation 10 is
>>> NOT already being processed
>>> 112
>>> 113 |5754|10:10|new| ---- WARNING ----
>>> 114 |5754|10:10|new| 2010-04-01
>> 15:37:39|5754|10:10|new|utils.pm:get_request_info(5218)|preferredimageid
>>> is not set for computer id=3
>>> 115 |5754|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 116 |5754|10:10|new| (-1) utils.pm, get_request_info (line: 5218)
>>> 117 |5754|10:10|new| (-2) vcld, main (line: 280)
>>> 118
>>> 119 2010-04-01 15:37:39|5754|10:10|new|utils.pm:get_request_info(5354)|standalone
>>> affiliation found: Local
>>> 120 2010-04-01
>> 15:37:39|5754|10:10|new|utils.pm:get_management_node_info(6815)|management
>>> node info retrieved from database for colbert
>>> 121 2010-04-01 15:37:39|5754|10:10|new|vcld:main(281)|retrieved request
>>> information from database
>>> 122 2010-04-01
>>> 15:37:39|5754|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
>>> true: parent reservation ID for this request: 10
>>> 123 2010-04-01 15:37:39|5754|10:10|new|utils.pm:update_request_state(2186)|request
>>> 10 state updated to: pending, laststate to: new
>>> 124 2010-04-01 15:37:39|5754|10:10|new|utils.pm:insertloadlog(4710)|inserted
>>> computer=3, begin, beginning to process, state is new
>>> 125 2010-04-01 15:37:39|5754|10:10|new|vcld:make_new_child(509)|loaded
>>> VCL::new module
>>> 126 2010-04-01 15:37:39|12726|10:10|new|vcld:make_new_child(582)|vcld
>>> environment variable set to 0 for this process
>>> 127 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
>>> called, class=VCL::new
>>> 128 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::new
>>> object created
>>> 129 2010-04-01 15:37:39|5754|10:10|new|vcld:make_new_child(566)|current
>>> number of forked kids: 2
>>> 130 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(85)|obtained
>>> a database handle for this state process, stored as $ENV{dbh}
>>> 131 2010-04-01 15:37:39|12726|10:10|new|State.pm:check_image_os(820)|no
>>> corrections need to be made, not an imaging request, returning 1
>>> 132 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
>>> true: parent reservation ID for this request: 10
>>> 133 2010-04-01
>> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7863)|reservation
>>> count: 1
>>> 134 2010-04-01
>> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7882)|PARENTIMAGE:
>>> 1
>>> 135 2010-04-01
>> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7883)|SUBIMAGE:
>>> 0
>>> 136 2010-04-01
>> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7901)|renamed
>>> process to 'vcld VCL::new 10:10 new'
>>> 137 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
>>> true: parent reservation ID for this request: 10
>>> 138 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
>>> true: parent reservation ID for this request: 10
>>> 139 2010-04-01
>>> 15:37:39|12726|10:10|new|State.pm:initialize(121)|attempting to load
>>> provisioning module: VCL::Module::Provisioning::esx
>>> 140 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx
>>> vmware toolkit root path found: /usr/lib/vmware-vcli/apps
>>> 141 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(140)|vmware
>>> ESX module initialized
>>> 142 2010-04-01
>>> 15:37:39|12726|10:10|new|State.pm:initialize(127)|VCL::Module::Provisioning::esx
>>> module loaded
>>> 143 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
>>> called, class=VCL::Module::Provisioning::esx
>>> 144 2010-04-01
>>> 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::Provisioning::esx
>>> object created
>>> 145 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx
>>> vmware toolkit root path found: /usr/lib/vmware-vcli/apps
>>> 146 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(140)|vmware
>>> ESX module initialized
>>> 147 2010-04-01
>>> 15:37:39|12726|10:10|new|State.pm:initialize(131)|VCL::Module::Provisioning::esx
>>> provisioner object created
>>> 148 2010-04-01
>>> 15:37:39|12726|10:10|new|State.pm:initialize(145)|attempting to load OS
>>> module: VCL::Module::OS::Linux
>>> 149 2010-04-01
>>> 15:37:39|12726|10:10|new|State.pm:initialize(151)|VCL::Module::OS::Linux
>>> module loaded
>>> 150 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
>>> called, class=VCL::Module::OS::Linux
>>> 151 2010-04-01
>>> 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::OS::Linux object
>>> created
>>> 152 2010-04-01
>>> 15:37:39|12726|10:10|new|State.pm:initialize(154)|VCL::Module::OS::Linux OS
>>> object created
>>> 153 2010-04-01
>>> 15:37:39|12726|10:10|new|State.pm:initialize(169)|returning 1
>>> 154 2010-04-01
>>> 15:37:39|12726|10:10|new|vcld:make_new_child(591)|VCL::new object created
>>> and initialized
>>> 155 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
>>> true: parent reservation ID for this request: 10
>>> 156 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
>>> to retrieve current state of computer esx3-basewinxp-v0 from the database
>>> 157 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
>>> current state of computer esx3-basewinxp-v0 from the database: available
>>> 158 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
>>> updated: $self->request_data->{reservation}{10}{computer}{state}{name}
>>> 159 |12726|10:10|new| computer_state_name = available
>>> 160
>>> 161 |12726|10:10|new| ---- WARNING ----
>>> 162 |12726|10:10|new| 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
>>> data has not been initialized for get_computer_preferredimage_id:
>>> $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
>>> 163 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 164 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
>>> 165 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
>>> 166 |12726|10:10|new| (-3) new.pm, process (line: 106)
>>> 167 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
>>> 168 |12726|10:10|new| (-5) vcld, main (line: 341)
>>> 169
>>> 170
>>> 171 |12726|10:10|new| ---- WARNING ----
>>> 172 |12726|10:10|new| 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
>>> data has not been initialized for get_computer_preferredimage_name:
>>> $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
>>> 173 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 174 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
>>> 175 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
>>> 176 |12726|10:10|new| (-3) new.pm, process (line: 107)
>>> 177 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
>>> 178 |12726|10:10|new| (-5) vcld, main (line: 341)
>>> 179
>>> 180 2010-04-01
>> 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
>>> variable: 1
>>> 181 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(134)|reservation
>>> is parent = 1
>>> 182 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(135)|preload
>>> only = 0
>>> 183 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(136)|originating
>>> request state = new
>>> 184 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(137)|originating
>>> request laststate = new
>>> 185 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(138)|originating
>>> computer state = available
>>> 186 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(139)|originating
>>> computer type = virtualmachine
>>> 187 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
>>> true: parent reservation ID for this request: 10
>>> 188 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
>>> to retrieve current state of computer esx3-basewinxp-v0 from the database
>>> 189 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
>>> current state of computer esx3-basewinxp-v0 from the database: available
>>> 190 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
>>> updated: $self->request_data->{reservation}{10}{computer}{state}{name}
>>> 191 |12726|10:10|new| computer_state_name = available
>>> 192
>>> 193 |12726|10:10|new| ---- WARNING ----
>>> 194 |12726|10:10|new| 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
>>> data has not been initialized for get_computer_preferredimage_id:
>>> $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
>>> 195 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 196 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
>>> 197 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
>>> 198 |12726|10:10|new| (-3) new.pm, computer_not_being_used (line: 750)
>>> 199 |12726|10:10|new| (-4) new.pm, process (line: 170)
>>> 200 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
>>> 201 |12726|10:10|new| (-6) vcld, main (line: 341)
>>> 202
>>> 203
>>> 204 |12726|10:10|new| ---- WARNING ----
>>> 205 |12726|10:10|new| 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
>>> data has not been initialized for get_computer_preferredimage_name:
>>> $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
>>> 206 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 207 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
>>> 208 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
>>> 209 |12726|10:10|new| (-3) new.pm, computer_not_being_used (line: 751)
>>> 210 |12726|10:10|new| (-4) new.pm, process (line: 170)
>>> 211 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
>>> 212 |12726|10:10|new| (-6) vcld, main (line: 341)
>>> 213
>>> 214 2010-04-01
>> 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
>>> variable: 1
>>> 215 2010-04-01
>> 15:37:39|12726|10:10|new|new.pm:computer_not_being_used(788)|esx3-basewinxp-v0
>>> state is available
>>> 216 2010-04-01
>> 15:37:39|12726|10:10|new|new.pm:computer_not_being_used(798)|esx3-basewinxp-v0
>>> is available, its state is available
>>> 217 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(170)|esx3-basewinxp-v0
>>> is not being used
>>> 218 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
>>> true: parent reservation ID for this request: 10
>>> 219 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
>>> to retrieve current state of computer esx3-basewinxp-v0 from the database
>>> 220 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
>>> current state of computer esx3-basewinxp-v0 from the database: available
>>> 221 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
>>> updated: $self->request_data->{reservation}{10}{computer}{state}{name}
>>> 222 |12726|10:10|new| computer_state_name = available
>>> 223
>>> 224 |12726|10:10|new| ---- WARNING ----
>>> 225 |12726|10:10|new| 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
>>> data has not been initialized for get_computer_preferredimage_id:
>>> $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
>>> 226 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 227 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
>>> 228 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
>>> 229 |12726|10:10|new| (-3) new.pm, reload_image (line: 480)
>>> 230 |12726|10:10|new| (-4) new.pm, process (line: 266)
>>> 231 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
>>> 232 |12726|10:10|new| (-6) vcld, main (line: 341)
>>> 233
>>> 234
>>> 235 |12726|10:10|new| ---- WARNING ----
>>> 236 |12726|10:10|new| 2010-04-01
>>> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
>>> data has not been initialized for get_computer_preferredimage_name:
>>> $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
>>> 237 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 238 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
>>> 239 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
>>> 240 |12726|10:10|new| (-3) new.pm, reload_image (line: 481)
>>> 241 |12726|10:10|new| (-4) new.pm, process (line: 266)
>>> 242 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
>>> 243 |12726|10:10|new| (-6) vcld, main (line: 341)
>>> 244
>>> 245 2010-04-01
>> 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
>>> variable: 1
>>> 246 2010-04-01 15:37:39|12726|10:10|new|new.pm:reload_image(514)|calling
>>> VCL::Module::Provisioning::esx->node_status()
>>> 247 2010-04-01 15:37:39|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
>>> computer=3, statuscheck, checking status of node
>>> 248 $VAR1 = bless( {
>>> 249 'data' => bless( do{\(my $o = 1)},
>>> 'VCL::DataStructure' ),
>>> 250 'os' => bless( {
>>> 251 'provisioner' => $VAR1,
>>> 252 'data' => $VAR1->{'data'}
>>> 253 }, 'VCL::Module::OS::Linux' )
>>> 254 }, 'VCL::Module::Provisioning::esx' );
>>> 255 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(787)|Entering
>>> node_status, checking status of esx3-basewinxp-v0
>>> 256 2010-04-01
>> 15:37:39|12726|10:10|new|esx.pm:node_status(788)|request_for_imaging:
>>> 0
>>> 257 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(789)|requeseted
>>> image name: esx3-basewinxp-v0
>>> 258 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(819)|checking
>>> if esx3-basewinxp-v0 is pingable
>>> 259 2010-04-01 15:37:40|12726|10:10|new|esx.pm:node_status(825)|esx3-basewinxp-
>> v0
>>> is not pingable (0)
>>> 260 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(560)|node_status
>>> returned a scalar: RELOAD
>>> 261 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(602)|node
>>> status is RELOAD, esx3-basewinxp-v0 will be reloaded
>>> 262 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
>>> computer=3, loadimageblade, esx3-basewinxp-v0 must be reloaded with
>>> esx3-basewinxp-v0
>>> 263 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(616)|calling
>>> VCL::Module::Provisioning::esx->does_image_exist()
>>> 264 2010-04-01
>> 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on 192.168.129.37:
>>> 265 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
>>> 192.168.129.37 'ls -1 /install/image/vmware_images/golden 2>&1' 2>&1
>>> 266 2010-04-01
>> 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
>>> output:
>>> 267 |12726|10:10|new| esx3-basewinxp-v0
>>> 268 2010-04-01 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
>>> command executed on 192.168.129.37, returning (0, "esx3-basewinxp-v0")
>>> 269 2010-04-01 15:37:40|12726|10:10|new|esx.pm:does_image_exist(917)|image
>>> esx3-basewinxp-v0 exists
>>> 270 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(619)|esx3-
>> basewinxp-v0
>>> exists on this management node
>>> 271 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
>>> computer=3, doesimageexists, confirmed image exists
>>> 272 2010-04-01
>> 15:37:40|12726|10:10|new|utils.pm:update_computer_state(2228)|computer
>>> 3 state updated to: reloading
>>> 273 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(652)|computer
>>> esx3-basewinxp-v0 state set to reloading
>>> 274 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
>>> computer=3, info, computer state updated to reloading
>>> 275 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(663)|calling
>>> VCL::Module::Provisioning::esx->load() subroutine
>>> 276 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
>>> computer=3, info, calling VCL::Module::Provisioning::esx->load() subroutine
>>> 277 2010-04-01 15:37:40|12726|10:10|new|esx.pm:
>>> load(165)|****************************************************
>>> 278
>>> 279 |12726|10:10|new| ---- WARNING ----
>>> 280 |12726|10:10|new| 2010-04-01
>>> 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
>>> data has not been initialized for get_computer_eth0_mac_address:
>>> $self->request_data->{reservation}{10}{computer}{eth0macaddress}
>>> 281 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 282 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
>>> 283 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
>>> 284 |12726|10:10|new| (-3) esx.pm, load (line: 181)
>>> 285 |12726|10:10|new| (-4) new.pm, reload_image (line: 665)
>>> 286 |12726|10:10|new| (-5) new.pm, process (line: 266)
>>> 287 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
>>> 288
>>> 289
>>> 290 |12726|10:10|new| ---- WARNING ----
>>> 291 |12726|10:10|new| 2010-04-01
>>> 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
>>> data has not been initialized for get_computer_eth1_mac_address:
>>> $self->request_data->{reservation}{10}{computer}{eth1macaddress}
>>> 292 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 293 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
>>> 294 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
>>> 295 |12726|10:10|new| (-3) esx.pm, load (line: 182)
>>> 296 |12726|10:10|new| (-4) new.pm, reload_image (line: 665)
>>> 297 |12726|10:10|new| (-5) new.pm, process (line: 266)
>>> 298 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
>>> 299
>>> 300 2010-04-01 15:37:40|12726|10:10|new|esx.pm:load(193)|Calling
>>> get_vmware_host_info
>>> 301 No recipient addresses found in header
>>> 302 2010-04-01 15:37:41|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
>>> Sending mail To: , PROBLEM -- esx.pm
>>> 303
>>> 304 |12726|10:10|new| ---- CRITICAL ----
>>> 305 |12726|10:10|new| 2010-04-01
>> 15:37:40|12726|10:10|new|esx.pm:get_vmware_host_info(1012)|AMIT
>>> DEBUGGING: esx35-hypervisor-1.odu.edu
>>> 306 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 307 |12726|10:10|new| (-1) esx.pm, get_vmware_host_info (line: 1012)
>>> 308 |12726|10:10|new| (-2) esx.pm, load (line: 194)
>>> 309 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
>>> 310 |12726|10:10|new| (-4) new.pm, process (line: 266)
>>> 311 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
>>> 312 |12726|10:10|new| (-6) vcld, main (line: 341)
>>> 313
>>> 314 2010-04-01 15:37:41|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 3/3: executing SSH command on esx35-hypervisor-1:
>>> 315 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'netstat -an' 2>&1
>>> 316
>>> 317 |12719|9:9|deleted| ---- WARNING ----
>>> 318 |12719|9:9|deleted| 2010-04-01
>> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 3/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
>>> status: 255, SSH exits with the exit status of the remote command or with
>>> 255 if an error occurred, output:
>>> 319 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
>>> found
>>> 320 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 321 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 322 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
>>> 323 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
>>> 324 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 325 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 326 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 327
>>> 328
>>> 329 |12719|9:9|deleted| ---- WARNING ----
>>> 330 |12719|9:9|deleted| 2010-04-01
>> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
>>> to run SSH command after 3 attempts, command: /usr/bin/ssh -i
>>> /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'netstat -an' 2>&1,
>>> exit status: 255, output:
>>> 331 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
>>> found
>>> 332 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 333 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
>>> 334 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
>>> 335 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
>>> 336 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 337 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 338 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 339
>>> 340 2010-04-01
>> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on esx35-hypervisor-1:
>>> 341 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
>>> /etc/ssh/external_sshd_config"' 2>&1
>>> 342
>>> 343 |12719|9:9|deleted| ---- WARNING ----
>>> 344 |12719|9:9|deleted| 2010-04-01
>> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 1/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx
>>> "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255, SSH
>>> exits with the exit status of the remote command or with 255 if an error
>>> occurred, output:
>>> 345 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not
>>> found
>>> 346 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 347 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 348 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line: 913)
>>> 349 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
>>> 350 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
>>> 351 |12719|9:9|deleted| (-5) reclaim.pm, process (line: 173)
>>> 352 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
>>> 353
>>> 354 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 2 seconds before making next SSH attempt
>>> 355 2010-04-01 15:37:43|12726|10:10|new|esx.pm:get_vmware_host_info(1019)|host
>>> info output for esx35-hypervisor-1
>>> 356 |12726|10:10|new| Host Information
>>> 357 |12726|10:10|new| Host Name: esx35-hypervisor-1.odu.edu
>>> 358 2010-04-01 15:37:43|12726|10:10|new|esx.pm:get_vmware_host_info(1025)|found
>>> hostname_value= esx35-hypervisor-1.odu.edu
>>> 359 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(197)|Collected
>>> esx35-hypervisor-1.odu.edu for vmware host name
>>> 360 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(210)|DATASTORE IP
>>> is 192.168.129.37 and DATASTORE_SHARE_PATH is /install/image/vmware_images
>>> 361 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(211)|Entered ESX
>>> module, loading esx3-basewinxp-v0 on esx3-basewinxp-v0 (on
>>> esx35-hypervisor-1.odu.edu) for reservation 10
>>> 362 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(212)|Datastore:
>>> 192.168.129.37:/install/image/vmware_images
>>> 363 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(223)|VM info
>>> command: /usr/lib/vmware-vcli/apps/vm/vminfo.pl --server
>>> 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --username root --password
>>> 'xxxxxxxxxxx'
>>> 364 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 2/3: executing SSH command on esx35-hypervisor-1:
>>> 365 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
>>> /etc/ssh/external_sshd_config"' 2>&1
>>> 366 2010-04-01 15:37:44|12726|10:10|new|esx.pm:load(226)|VM info
>>> output: Virtual Machine esx3-basewinxp-v0 not found.
>>> 367 2010-04-01
>> 15:37:44|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on 192.168.129.37:
>>> 368 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
>>> 192.168.129.37 'rm -rf /install/image/vmware_images/inuse/esx3-basewinxp-v0'
>>> 2>&1
>>> 369 2010-04-01 15:37:44|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:37:44
>>> 370
>>> 371 |12719|9:9|deleted| ---- WARNING ----
>>> 372 |12719|9:9|deleted| 2010-04-01
>> 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 2/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx
>>> "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255, SSH
>>> exits with the exit status of the remote command or with 255 if an error
>>> occurred, output:
>>> 373 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not
>>> found
>>> 374 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 375 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 376 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line: 913)
>>> 377 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
>>> 378 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
>>> 379 |12719|9:9|deleted| (-5) reclaim.pm, process (line: 173)
>>> 380 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
>>> 381
>>> 382 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 15 seconds before making next SSH attempt
>>> 383 2010-04-01
>> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
>>> output:
>>> 384 |12726|10:10|new| none
>>> 385 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
>>> command executed on 192.168.129.37, returning (0, "none")
>>> 386 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(262)|Removed old vm
>>> folder
>>> 387 2010-04-01
>> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on 192.168.129.37:
>>> 388 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
>>> 192.168.129.37 'mkdir /install/image/vmware_images/inuse/esx3-basewinxp-v0'
>>> 2>&1
>>> 389 2010-04-01
>> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
>>> output:
>>> 390 |12726|10:10|new| none
>>> 391 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
>>> command executed on 192.168.129.37, returning (0, "none")
>>> 392 2010-04-01
>> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on 192.168.129.37:
>>> 393 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
>>> 192.168.129.37 'cp
>>> /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk
>>> /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk'
>>> 2>&1
>>> 394 2010-04-01
>> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
>>> output:
>>> 395 |12726|10:10|new| none
>>> 396 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
>>> command executed on 192.168.129.37, returning (0, "none")
>>> 397 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(278)|COPIED VMDK
>>> SUCCESSFULLY
>>> 398 2010-04-01
>> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on 192.168.129.37:
>>> 399 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
>>> 192.168.129.37 'cp
>>> /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk
>>> /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk'
>>> 2>&1
>>> 400 2010-04-01 15:37:49|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:37:49
>>> 401 2010-04-01 15:37:54|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:37:54
>>> 402 2010-04-01 15:37:59|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 3/3: executing SSH command on esx35-hypervisor-1:
>>> 403 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
>>> /etc/ssh/external_sshd_config"' 2>&1
>>> 404 2010-04-01 15:37:59|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:37:59
>>> 405 2010-04-01
>> 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6262)|run_ssh_command
>>> output:
>>> 406 |12719|9:9|deleted| ash: pkill: not found
>>> 407 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6272)|SSH
>>> command executed on esx35-hypervisor-1, command:
>>> 408 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
>>> /etc/ssh/external_sshd_config"' 2>&1
>>> 409 |12719|9:9|deleted| returning (127, "ash: pkill: not found")
>>> 410 2010-04-01
>>> 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(916)|output detected:
>>> ash: pkill: not found
>>> 411 2010-04-01
>>> 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(921)|ext_sshd on
>>> esx35-hypervisor-1 stopped
>>> 412 2010-04-01
>>> 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(884)|stopped external sshd
>>> 413 2010-04-01
>>> 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(888)|access has been
>>> revoked to esx35-hypervisor-1
>>> 414 2010-04-01
>> 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on esx35-hypervisor-1:
>>> 415 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
>>> 416
>>> 417 |12719|9:9|deleted| ---- WARNING ----
>>> 418 |12719|9:9|deleted| 2010-04-01
>> 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 1/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
>>> admin, exit status: 255, SSH exits with the exit status of the remote
>>> command or with 255 if an error occurred, output:
>>> 419 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/passwd'
>>> 420 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/shadow'
>>> 421 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/group'
>>> 422 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/gshadow'
>>> 423 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 424 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 425 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
>>> 426 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 427 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 428 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 429 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 430
>>> 431 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 2 seconds before making next SSH attempt
>>> 432 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 2/3: executing SSH command on esx35-hypervisor-1:
>>> 433 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
>>> 434
>>> 435 |12719|9:9|deleted| ---- WARNING ----
>>> 436 |12719|9:9|deleted| 2010-04-01
>> 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 2/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
>>> admin, exit status: 255, SSH exits with the exit status of the remote
>>> command or with 255 if an error occurred, output:
>>> 437 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/passwd'
>>> 438 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/shadow'
>>> 439 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/group'
>>> 440 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/gshadow'
>>> 441 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 442 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 443 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
>>> 444 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 445 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 446 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 447 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 448
>>> 449 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 15 seconds before making next SSH attempt
>>> 450 2010-04-01 15:38:04|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:38:04
>>> 451 2010-04-01 15:38:09|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:38:09
>>> 452 2010-04-01 15:38:14|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:38:14
>>> 453 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 3/3: executing SSH command on esx35-hypervisor-1:
>>> 454 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
>>> 455
>>> 456 |12719|9:9|deleted| ---- WARNING ----
>>> 457 |12719|9:9|deleted| 2010-04-01
>> 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 3/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
>>> admin, exit status: 255, SSH exits with the exit status of the remote
>>> command or with 255 if an error occurred, output:
>>> 458 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/passwd'
>>> 459 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/shadow'
>>> 460 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/group'
>>> 461 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/gshadow'
>>> 462 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 463 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 464 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
>>> 465 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 466 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 467 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 468 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 469
>>> 470
>>> 471 |12719|9:9|deleted| ---- WARNING ----
>>> 472 |12719|9:9|deleted| 2010-04-01
>> 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
>>> to run SSH command after 3 attempts, command: /usr/bin/ssh -i
>>> /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel
>>> admin' 2>&1, exit status: 255, output:
>>> 473 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/passwd'
>>> 474 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/shadow'
>>> 475 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/group'
>>> 476 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/gshadow'
>>> 477 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 478 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
>>> 479 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
>>> 480 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 481 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 482 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 483 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 484
>>> 485 2010-04-01
>> 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on esx35-hypervisor-1:
>>> 486 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
>>> 487
>>> 488 |12719|9:9|deleted| ---- WARNING ----
>>> 489 |12719|9:9|deleted| 2010-04-01
>> 15:38:18|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 1/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
>>> admin, exit status: 255, SSH exits with the exit status of the remote
>>> command or with 255 if an error occurred, output:
>>> 490 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/passwd'
>>> 491 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/shadow'
>>> 492 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/group'
>>> 493 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/gshadow'
>>> 494 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 495 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 496 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
>>> 497 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 498 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 499 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 500 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 501
>>> 502 2010-04-01 15:38:18|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 2 seconds before making next SSH attempt
>>> 503 2010-04-01 15:38:19|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:38:19
>>> 504 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 2/3: executing SSH command on esx35-hypervisor-1:
>>> 505 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
>>> 506
>>> 507 |12719|9:9|deleted| ---- WARNING ----
>>> 508 |12719|9:9|deleted| 2010-04-01
>> 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 2/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
>>> admin, exit status: 255, SSH exits with the exit status of the remote
>>> command or with 255 if an error occurred, output:
>>> 509 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/passwd'
>>> 510 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/shadow'
>>> 511 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/group'
>>> 512 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/gshadow'
>>> 513 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 514 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 515 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
>>> 516 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 517 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 518 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 519 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 520
>>> 521 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 15 seconds before making next SSH attempt
>>> 522 2010-04-01 15:38:25|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:38:24
>>> 523 2010-04-01 15:38:30|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:38:30
>>> 524 2010-04-01 15:38:35|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:38:35
>>> 525 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 3/3: executing SSH command on esx35-hypervisor-1:
>>> 526 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
>>> 527
>>> 528 |12719|9:9|deleted| ---- WARNING ----
>>> 529 |12719|9:9|deleted| 2010-04-01
>> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 3/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
>>> admin, exit status: 255, SSH exits with the exit status of the remote
>>> command or with 255 if an error occurred, output:
>>> 530 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/passwd'
>>> 531 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/shadow'
>>> 532 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/group'
>>> 533 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/gshadow'
>>> 534 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 535 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 536 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
>>> 537 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 538 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 539 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 540 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 541
>>> 542
>>> 543 |12719|9:9|deleted| ---- WARNING ----
>>> 544 |12719|9:9|deleted| 2010-04-01
>> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
>>> to run SSH command after 3 attempts, command: /usr/bin/ssh -i
>>> /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel
>>> admin' 2>&1, exit status: 255, output:
>>> 545 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/passwd'
>>> 546 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/shadow'
>>> 547 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/group'
>>> 548 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
>>> 'admin' in '/etc/gshadow'
>>> 549 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 550 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
>>> 551 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
>>> 552 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 553 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 554 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 555 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 556
>>> 557 2010-04-01
>> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on esx35-hypervisor-1:
>>> 558 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
>>> /etc/ssh/external_sshd_config' 2>&1
>>> 559
>>> 560 |12719|9:9|deleted| ---- WARNING ----
>>> 561 |12719|9:9|deleted| 2010-04-01
>> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
>>> "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
>>> exits with the exit status of the remote command or with 255 if an error
>>> occurred, output:
>>> 562 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
>>> /etc/ssh/external_sshd_config: No such file or directory
>>> 563 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 564 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 565 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
>>> 566 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 567 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 568 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 569 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 570
>>> 571 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 2 seconds before making next SSH attempt
>>> 572 2010-04-01 15:38:37|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 2/3: executing SSH command on esx35-hypervisor-1:
>>> 573 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
>>> /etc/ssh/external_sshd_config' 2>&1
>>> 574
>>> 575 |12719|9:9|deleted| ---- WARNING ----
>>> 576 |12719|9:9|deleted| 2010-04-01
>> 15:38:38|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
>>> "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
>>> exits with the exit status of the remote command or with 255 if an error
>>> occurred, output:
>>> 577 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
>>> /etc/ssh/external_sshd_config: No such file or directory
>>> 578 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 579 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 580 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
>>> 581 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 582 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 583 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 584 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 585
>>> 586 2010-04-01 15:38:38|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 15 seconds before making next SSH attempt
>>> 587 2010-04-01 15:38:40|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:38:40
>>> 588 2010-04-01 15:38:45|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:38:45
>>> 589 2010-04-01 15:38:50|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:38:50
>>> 590 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 3/3: executing SSH command on esx35-hypervisor-1:
>>> 591 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
>>> /etc/ssh/external_sshd_config' 2>&1
>>> 592
>>> 593 |12719|9:9|deleted| ---- WARNING ----
>>> 594 |12719|9:9|deleted| 2010-04-01
>> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
>>> "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
>>> exits with the exit status of the remote command or with 255 if an error
>>> occurred, output:
>>> 595 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
>>> /etc/ssh/external_sshd_config: No such file or directory
>>> 596 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 597 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 598 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
>>> 599 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 600 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 601 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 602 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 603
>>> 604
>>> 605 |12719|9:9|deleted| ---- WARNING ----
>>> 606 |12719|9:9|deleted| 2010-04-01
>> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
>>> to run SSH command after 3 attempts, command: /usr/bin/ssh -i
>>> /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^AllowUsers
>>> .*/d" /etc/ssh/external_sshd_config' 2>&1, exit status: 255, output:
>>> 607 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
>>> /etc/ssh/external_sshd_config: No such file or directory
>>> 608 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 609 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
>>> 610 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
>>> 611 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 612 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 613 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 614 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 615
>>> 616 No recipient addresses found in header
>>> 617 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:mail(1348)|SUCCESS
>>> -- Sending mail To: , PROBLEM -- Linux.pm
>>> 618
>>> 619 |12719|9:9|deleted| ---- CRITICAL ----
>>> 620 |12719|9:9|deleted| 2010-04-01
>>> 15:38:53|12719|9:9|deleted|Linux.pm:delete_user(566)|failed to add
>>> AllowUsers admin to external_sshd_config
>>> 621 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 622 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 566)
>>> 623 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
>>> 624 |12719|9:9|deleted| (-3) reclaim.pm, process (line: 173)
>>> 625 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
>>> 626 |12719|9:9|deleted| (-5) vcld, main (line: 341)
>>> 627
>>> 628 2010-04-01
>> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on esx35-hypervisor-1:
>>> 629 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
>>> 630
>>> 631 |12719|9:9|deleted| ---- WARNING ----
>>> 632 |12719|9:9|deleted| 2010-04-01
>> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
>>> .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
>>> remote command or with 255 if an error occurred, output:
>>> 633 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
>>> No such file or directory
>>> 634 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 635 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 636 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
>>> 637 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 638 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 639 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 640 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 641
>>> 642 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 2 seconds before making next SSH attempt
>>> 643 2010-04-01 15:38:55|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:38:55
>>> 644 2010-04-01 15:38:55|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 2/3: executing SSH command on esx35-hypervisor-1:
>>> 645 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
>>> 646
>>> 647 |12719|9:9|deleted| ---- WARNING ----
>>> 648 |12719|9:9|deleted| 2010-04-01
>> 15:38:56|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
>>> .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
>>> remote command or with 255 if an error occurred, output:
>>> 649 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
>>> No such file or directory
>>> 650 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 651 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 652 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
>>> 653 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 654 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 655 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 656 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 657
>>> 658 2010-04-01 15:38:56|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
>>> for 15 seconds before making next SSH attempt
>>> 659 2010-04-01 15:39:00|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:00
>>> 660 2010-04-01 15:39:05|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:05
>>> 661 2010-04-01 15:39:10|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:10
>>> 662 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
>>> 3/3: executing SSH command on esx35-hypervisor-1:
>>> 663 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
>>> -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
>>> 664
>>> 665 |12719|9:9|deleted| ---- WARNING ----
>>> 666 |12719|9:9|deleted| 2010-04-01
>> 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
>>> 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
>>> .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
>>> remote command or with 255 if an error occurred, output:
>>> 667 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
>>> No such file or directory
>>> 668 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 669 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
>>> 670 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
>>> 671 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 672 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 673 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 674 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 675
>>> 676
>>> 677 |12719|9:9|deleted| ---- WARNING ----
>>> 678 |12719|9:9|deleted| 2010-04-01
>> 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
>>> to run SSH command after 3 attempts, command: /usr/bin/ssh -i
>>> /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^admin
>>> .*/d" /etc/sudoers' 2>&1, exit status: 255, output:
>>> 679 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
>>> No such file or directory
>>> 680 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 681 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
>>> 682 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
>>> 683 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
>>> 684 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
>>> 685 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
>>> 686 |12719|9:9|deleted| (-6) vcld, main (line: 341)
>>> 687
>>> 688 No recipient addresses found in header
>>> 689 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:mail(1348)|SUCCESS
>>> -- Sending mail To: , PROBLEM -- Linux.pm
>>> 690
>>> 691 |12719|9:9|deleted| ---- CRITICAL ----
>>> 692 |12719|9:9|deleted| 2010-04-01
>>> 15:39:11|12719|9:9|deleted|Linux.pm:delete_user(578)|failed to clear admin
>>> from /etc/sudoers
>>> 693 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
>>> 694 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 578)
>>> 695 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
>>> 696 |12719|9:9|deleted| (-3) reclaim.pm, process (line: 173)
>>> 697 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
>>> 698 |12719|9:9|deleted| (-5) vcld, main (line: 341)
>>> 699
>>> 700 2010-04-01 15:39:11|12719|9:9|deleted|Linux.pm:sanitize(852)|users
>>> have been deleted from esx35-hypervisor-1
>>> 701 2010-04-01 15:39:11|12719|9:9|deleted|reclaim.pm:process(173)|esx35-
>> hypervisor-1
>>> has been sanitized
>>> 702 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8018)|called
>>> from VCL::reclaim::process(271)
>>> 703 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8054)|parent:
>>> parent reservation ID for this request: 9
>>> 704 2010-04-01
>> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
>>> entry not inserted, reservation has been deleted
>>> 705 2010-04-01
>> 15:39:11|12719|9:9|deleted|utils.pm:update_request_state(2186)|request
>>> 9 state updated to: complete, laststate to: deleted
>>> 706 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8078)|req=9:
>>> request state changed: deleted->complete, laststate: reserved->deleted
>>> 707 2010-04-01
>> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
>>> entry not inserted, reservation has been deleted
>>> 708 2010-04-01
>> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
>>> entry not inserted, reservation has been deleted
>>> 709 2010-04-01
>> 15:39:11|12719|9:9|deleted|utils.pm:update_computer_state(2228)|computer
>>> 2 state updated to: available
>>> 710 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8096)|req=9:
>>> computer esx35-hypervisor-1 state changed: available->available
>>> 711 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8106)|req=9:
>>> log table id=10 will not be updated
>>> 712 2010-04-01
>> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
>>> entry not inserted, reservation has been deleted
>>> 713 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8122)|req=9:
>>> process exiting
>>> 714 2010-04-01
>>> 15:39:11|12719|9:9|deleted|State.pm:DESTROY(905)|destructor called,
>>> ref($self)=VCL::reclaim
>>> 715 2010-04-01
>> 15:39:11|12719|9:9|deleted|utils.pm:delete_computerloadlog_reservation(7551)|removing
>>> computerloadlog entries matching loadstate = begin
>>> 716 2010-04-01
>> 15:39:11|12719|9:9|deleted|utils.pm:delete_computerloadlog_reservation(7598)|deleted
>>> rows from computerloadlog for reservation id=9
>>> 717 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(912)|removed
>>> computerloadlog rows with loadstate=begin for reservation
>>> 718 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(924)|number
>>> of database handles state process created: 1
>>> 719 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(933)|process
>>> has a database handle stored in $ENV{dbh}, attempting disconnect
>>> 720 2010-04-01
>>> 15:39:11|12719|9:9|deleted|State.pm:DESTROY(935)|$ENV{dbh}: database
>>> disconnect successful
>>> 721 2010-04-01
>>> 15:39:11|12719|9:9|deleted|State.pm:DESTROY(949)|VCL::reclaim process 12719
>>> exiting
>>> 722 2010-04-01 15:39:11|5754|vcld:REAPER(744)|VCL process exited for
>>> reservation 9
>>> 723 2010-04-01 15:39:11|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:11
>>> 724 2010-04-01 15:39:11|5754|9:9|complete|vcld:main(250)|request deleted
>>> 725 2010-04-01 15:39:16|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:16
>>> 726 2010-04-01 15:39:21|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:21
>>> 727 2010-04-01 15:39:26|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:26
>>> 728 2010-04-01 15:39:31|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:31
>>> 729 2010-04-01 15:39:36|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:36
>>> 730 2010-04-01 15:39:42|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:42
>>> 731 2010-04-01 15:39:47|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:47
>>> 732 2010-04-01 15:39:52|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:52
>>> 733 2010-04-01
>> 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
>>> output:
>>> 734 |12726|10:10|new| none
>>> 735 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
>>> command executed on 192.168.129.37, returning (0, "none")
>>> 736 2010-04-01
>> 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
>>> SSH command on 192.168.129.37:
>>> 737 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
>>> 192.168.129.37 'grep adapterType
>>> /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk
>>> 2>&1' 2>&1
>>> 738 2010-04-01
>> 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
>>> output:
>>> 739 |12726|10:10|new| ddb.adapterType = "buslogic"
>>> 740 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
>>> command executed on 192.168.129.37, returning (0, "ddb.adapterType =
>>> "buslogic"")
>>> 741 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(311)|adapter=
>>> buslogic
>>> 742 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(342)|eth0 MAC
>>> address set for vcl assigned
>>> 743 Use of uninitialized value in concatenation (.) or string at
>>> 744 /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pmline 343 (#1)
>>> 745 (W uninitialized) An undefined value was used as if it were
>>> already
>>> 746 defined. It was interpreted as a "" or a 0, but maybe it was a
>>> mistake.
>>> 747 To suppress this warning assign a defined value to your
>>> variables.
>>> 748
>>> 749 To help you figure out what was undefined, perl tells you what
>>> operation
>>> 750 you used the undefined value in. Note, however, that perl
>>> optimizes your
>>> 751 program and the operation displayed in the warning may not
>>> necessarily
>>> 752 appear literally in your program. For example, "that $foo" is
>>> 753 usually optimized into "that " . $foo, and the warning will
>>> refer to
>>> 754 the concatenation (.) operator, even though there is no . in
>>> your
>>> 755 program.
>>> 756
>>> 757
>>> 758 |12726|10:10|new| ---- WARNING ----
>>> 759 |12726|10:10|new| 2010-04-01
>>> 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
>>> value in concatenation (.) or string at
>>> /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 343.
>>> 760 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 761 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
>>> 762 |12726|10:10|new| (-2) esx.pm, load (line: 343)
>>> 763 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
>>> 764 |12726|10:10|new| (-4) new.pm, process (line: 266)
>>> 765 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
>>> 766 |12726|10:10|new| (-6) vcld, main (line: 341)
>>> 767
>>> 768
>>> 769 |12726|10:10|new| ---- WARNING ----
>>> 770 |12726|10:10|new| 2010-04-01
>>> 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
>>> value in concatenation (.) or string at
>>> /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 353.
>>> 771 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 772 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
>>> 773 |12726|10:10|new| (-2) esx.pm, load (line: 353)
>>> 774 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
>>> 775 |12726|10:10|new| (-4) new.pm, process (line: 266)
>>> 776 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
>>> 777 |12726|10:10|new| (-6) vcld, main (line: 341)
>>> 778
>>> 779 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(353)|eth1 MAC
>>> address set for vcl assigned
>>> 780
>>> 781 |12726|10:10|new| ---- WARNING ----
>>> 782 |12726|10:10|new| 2010-04-01
>>> 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
>>> value in concatenation (.) or string at
>>> /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 354.
>>> 783 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 784 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
>>> 785 |12726|10:10|new| (-2) esx.pm, load (line: 354)
>>> 786 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
>>> 787 |12726|10:10|new| (-4) new.pm, process (line: 266)
>>> 788 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
>>> 789 |12726|10:10|new| (-6) vcld, main (line: 341)
>>> 790
>>> 791 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(376)|wrote vmxarray
>>> to /tmp/esx3-basewinxp-v0.vmx
>>> 792 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_scp_command(6403)|attempt
>>> 1/3: executing SCP command: /usr/bin/scp -B -i /etc/vcl/vcl.key -P 22 -p -r
>>> /tmp/esx3-basewinxp-v0.vmx 192.168.129.37:/install/image/vmware_images/inuse/esx3-
>> basewinxp-v0/esx3-basewinxp-v0.vmx
>>> 2>&1
>>> 793 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_scp_command(6453)|scp
>>> successful: attempt 1/3, exit status: 0, output: none
>>> 794 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(401)|Register
>>> Command: /usr/lib/vmware-vcli/apps/vm/vmregister.pl --server
>>> 'esx35-hypervisor-1' --username root --password 'xxxxxxxxxxx' --vmxpath
>>> '[VCL]/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmx' --operation register
>>> --vmname esx3-basewinxp-v0 --pool Resources --hostname '
>>> esx35-hypervisor-1.odu.edu' --datacenter 'ha-datacenter'
>>> 795 2010-04-01 15:39:57|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:39:57
>>> 796 2010-04-01 15:39:57|12726|10:10|new|esx.pm:load(404)|Registered:
>>> 797 |12726|10:10|new| SOAP Fault:
>>> 798 |12726|10:10|new| -----------
>>> 799 |12726|10:10|new| Fault string: fault.RestrictedVersion.summary
>>> 800 |12726|10:10|new| Fault detail: RestrictedVersionFault
>>> 801 2010-04-01 15:39:57|12726|10:10|new|esx.pm:load(413)|Power on
>>> command: /usr/lib/vmware-vcli/apps/vm/vmcontrol.pl --server
>>> 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --operation poweron
>>> --username root --password 'xxxxxxxxxxx'
>>> 802 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(416)|Powered on:
>>> Virtual Machine esx3-basewinxp-v0 not found.
>>> 803 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(505)|IP is known
>>> for esx3-basewinxp-v0
>>> 804 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(511)|Waiting for
>>> ssh to come up on esx3-basewinxp-v0
>>> 805 2010-04-01 15:40:00|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 1
>>> 806 2010-04-01 15:40:02|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:02
>>> 807 2010-04-01 15:40:06|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 2
>>> 808 2010-04-01 15:40:07|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:07
>>> 809 2010-04-01 15:40:12|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:12
>>> 810 2010-04-01 15:40:12|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 3
>>> 811 2010-04-01 15:40:17|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:17
>>> 812 2010-04-01 15:40:18|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 4
>>> 813 2010-04-01 15:40:22|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:22
>>> 814 2010-04-01 15:40:24|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 5
>>> 815 2010-04-01 15:40:27|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:27
>>> 816 2010-04-01 15:40:30|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 6
>>> 817 2010-04-01 15:40:32|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:32
>>> 818 2010-04-01 15:40:36|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 7
>>> 819 2010-04-01 15:40:37|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:37
>>> 820 2010-04-01 15:40:42|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:42
>>> 821 2010-04-01 15:40:42|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 8
>>> 822 2010-04-01 15:40:47|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:47
>>> 823 2010-04-01 15:40:48|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 9
>>> 824 2010-04-01 15:40:52|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:52
>>> 825 2010-04-01 15:40:54|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 10
>>> 826 2010-04-01 15:40:57|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:40:57
>>> 827 2010-04-01 15:41:00|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 11
>>> 828 2010-04-01 15:41:02|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:02
>>> 829 2010-04-01 15:41:06|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 12
>>> 830 2010-04-01 15:41:07|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:07
>>> 831 2010-04-01 15:41:12|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:12
>>> 832 2010-04-01 15:41:12|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 13
>>> 833 2010-04-01 15:41:17|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:17
>>> 834 2010-04-01 15:41:18|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 14
>>> 835 2010-04-01 15:41:22|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:22
>>> 836 2010-04-01 15:41:24|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 15
>>> 837 2010-04-01 15:41:27|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:27
>>> 838 2010-04-01 15:41:30|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 16
>>> 839 2010-04-01 15:41:32|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:32
>>> 840 2010-04-01 15:41:36|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 17
>>> 841 2010-04-01 15:41:37|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:37
>>> 842 2010-04-01 15:41:42|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:42
>>> 843 2010-04-01 15:41:42|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 18
>>> 844 2010-04-01 15:41:47|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:47
>>> 845 2010-04-01 15:41:48|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 19
>>> 846 2010-04-01 15:41:52|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:52
>>> 847 2010-04-01 15:41:54|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 20
>>> 848 2010-04-01 15:41:57|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:41:57
>>> 849 2010-04-01 15:42:01|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 21
>>> 850 2010-04-01 15:42:02|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:02
>>> 851 2010-04-01 15:42:07|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 22
>>> 852 2010-04-01 15:42:07|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:07
>>> 853 2010-04-01 15:42:12|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:12
>>> 854 2010-04-01 15:42:13|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 23
>>> 855 2010-04-01 15:42:18|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:18
>>> 856 2010-04-01 15:42:19|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 24
>>> 857 2010-04-01 15:42:23|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:23
>>> 858 2010-04-01 15:42:25|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 25
>>> 859 2010-04-01 15:42:28|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:28
>>> 860 2010-04-01 15:42:31|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 26
>>> 861 2010-04-01 15:42:33|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:33
>>> 862 2010-04-01 15:42:37|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 27
>>> 863 2010-04-01 15:42:38|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:38
>>> 864 2010-04-01 15:42:43|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 28
>>> 865 2010-04-01 15:42:43|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:43
>>> 866 2010-04-01 15:42:48|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:48
>>> 867 2010-04-01 15:42:49|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 29
>>> 868 2010-04-01 15:42:53|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:53
>>> 869 2010-04-01 15:42:55|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 30
>>> 870 2010-04-01 15:42:58|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:42:58
>>> 871 2010-04-01 15:43:01|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 31
>>> 872 2010-04-01 15:43:03|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:03
>>> 873 2010-04-01 15:43:07|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 32
>>> 874 2010-04-01 15:43:08|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:08
>>> 875 2010-04-01 15:43:13|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 33
>>> 876 2010-04-01 15:43:13|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:13
>>> 877 2010-04-01 15:43:18|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:18
>>> 878 2010-04-01 15:43:19|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 34
>>> 879 2010-04-01 15:43:23|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:23
>>> 880 2010-04-01 15:43:25|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 35
>>> 881 2010-04-01 15:43:28|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:28
>>> 882 2010-04-01 15:43:31|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 36
>>> 883 2010-04-01 15:43:33|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:33
>>> 884 2010-04-01 15:43:37|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 37
>>> 885 2010-04-01 15:43:38|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:38
>>> 886 2010-04-01 15:43:43|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 38
>>> 887 2010-04-01 15:43:43|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:43
>>> 888 2010-04-01 15:43:48|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:48
>>> 889 2010-04-01 15:43:49|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 39
>>> 890 2010-04-01 15:43:53|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:53
>>> 891 2010-04-01 15:43:55|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 40
>>> 892 2010-04-01 15:43:58|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:43:58
>>> 893 2010-04-01 15:44:01|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 41
>>> 894 2010-04-01 15:44:03|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:03
>>> 895 2010-04-01 15:44:07|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 42
>>> 896 2010-04-01 15:44:08|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:08
>>> 897 2010-04-01 15:44:13|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 43
>>> 898 2010-04-01 15:44:13|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:13
>>> 899 2010-04-01 15:44:18|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:18
>>> 900 2010-04-01 15:44:19|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 44
>>> 901 2010-04-01 15:44:23|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:23
>>> 902 2010-04-01 15:44:25|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 45
>>> 903 2010-04-01 15:44:29|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:28
>>> 904 2010-04-01 15:44:31|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 46
>>> 905 2010-04-01 15:44:34|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:34
>>> 906 2010-04-01 15:44:37|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 47
>>> 907 2010-04-01 15:44:39|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:39
>>> 908 2010-04-01 15:44:43|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 48
>>> 909 2010-04-01 15:44:44|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:44
>>> 910 2010-04-01 15:44:49|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:49
>>> 911 2010-04-01 15:44:49|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 49
>>> 912 2010-04-01 15:44:54|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:54
>>> 913 2010-04-01 15:44:55|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 50
>>> 914 2010-04-01 15:44:59|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:44:59
>>> 915 2010-04-01 15:45:01|12726|10:10|new|esx.pm:load(528)|going to sleep
>>> 5 seconds, waiting for computer to start SSH. Try 51
>>> 916 2010-04-01 15:45:04|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:45:04
>>> 917 No recipient addresses found in header
>>> 918 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
>>> Sending mail To: , PROBLEM -- esx.pm
>>> 919
>>> 920 |12726|10:10|new| ---- CRITICAL ----
>>> 921 |12726|10:10|new| 2010-04-01 15:45:07|12726|10:10|new|esx.pm:load(521)|waited
>>> acceptable amount of time for sshd to become active, please check
>>> esx3-basewinxp-v0 on esx35-hypervisor-1
>>> 922 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 923 |12726|10:10|new| (-1) esx.pm, load (line: 521)
>>> 924 |12726|10:10|new| (-2) new.pm, reload_image (line: 665)
>>> 925 |12726|10:10|new| (-3) new.pm, process (line: 266)
>>> 926 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
>>> 927 |12726|10:10|new| (-5) vcld, main (line: 341)
>>> 928
>>> 929
>>> 930 |12726|10:10|new| ---- WARNING ----
>>> 931 |12726|10:10|new| 2010-04-01
>> 15:45:08|12726|10:10|new|new.pm:reload_image(670)|esx3-basewinxp-v0
>>> failed to load on esx3-basewinxp-v0, returning
>>> 932 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 933 |12726|10:10|new| (-1) new.pm, reload_image (line: 670)
>>> 934 |12726|10:10|new| (-2) new.pm, process (line: 266)
>>> 935 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
>>> 936 |12726|10:10|new| (-4) vcld, main (line: 341)
>>> 937
>>> 938 2010-04-01 15:45:08|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
>>> computer=3, loadimagefailed, esx3-basewinxp-v0 failed to load on
>>> esx3-basewinxp-v0
>>> 939
>>> 940 |12726|10:10|new| ---- WARNING ----
>>> 941 |12726|10:10|new| 2010-04-01
>> 15:45:08|12726|10:10|new|new.pm:process(313)|failed
>>> to load esx3-basewinxp-v0 with esx3-basewinxp-v0
>>> 942 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 943 |12726|10:10|new| (-1) new.pm, process (line: 313)
>>> 944 |12726|10:10|new| (-2) vcld, make_new_child (line: 594)
>>> 945 |12726|10:10|new| (-3) vcld, main (line: 341)
>>> 946
>>> 947 2010-04-01
>>> 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
>>> to retrieve current state of computer esx3-basewinxp-v0 from the database
>>> 948 2010-04-01
>>> 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
>>> current state of computer esx3-basewinxp-v0 from the database: reloading
>>> 949 2010-04-01
>>> 15:45:08|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
>>> updated: $self->request_data->{reservation}{10}{computer}{state}{name}
>>> 950 |12726|10:10|new| computer_state_name = reloading
>>> 951 No recipient addresses found in header
>>> 952 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
>>> Sending mail To: , PROBLEM -- State.pm
>>> 953
>>> 954 |12726|10:10|new| ---- CRITICAL ----
>>> 955 |12726|10:10|new| 2010-04-01
>>> 15:45:08|12726|10:10|new|State.pm:reservation_failed(290)|reservation failed
>>> on esx3-basewinxp-v0: process failed after trying to load or make available
>>> 956 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
>>> 957 |12726|10:10|new| (-1) State.pm, reservation_failed (line: 290)
>>> 958 |12726|10:10|new| (-2) new.pm, process (line: 316)
>>> 959 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
>>> 960 |12726|10:10|new| (-4) vcld, main (line: 341)
>>> 961
>>> 962 2010-04-01 15:45:08|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
>>> computer=3, failed, process failed after trying to load or make available
>>> 963 2010-04-01
>>> 15:45:08|12726|10:10|new|State.pm:reservation_failed(293)|inserted
>>> computerloadlog entry
>>> 964 2010-04-01
>>> 15:45:08|12726|10:10|new|State.pm:reservation_failed(301)|updated log ending
>>> value to 'failed', logid=11
>>> 965 2010-04-01
>> 15:45:08|12726|10:10|new|utils.pm:update_computer_state(2228)|computer
>>> 3 state updated to: failed
>>> 966 2010-04-01
>>> 15:45:08|12726|10:10|new|State.pm:reservation_failed(312)|computer
>>> esx3-basewinxp-v0 (3) state set to failed
>>> 967 2010-04-01 15:45:08|12726|10:10|new|utils.pm:update_request_state(2186)|request
>>> 10 state updated to: failed, laststate to: new
>>> 968 2010-04-01
>>> 15:45:08|12726|10:10|new|State.pm:reservation_failed(325)|set request state
>>> to 'failed'/'new'
>>> 969 2010-04-01 15:45:08|12726|10:10|new|utils.pm:is_inblockrequest(6972)|zero
>>> rows were returned from database select
>>> 970 2010-04-01
>>> 15:45:08|12726|10:10|new|State.pm:reservation_failed(343)|esx3-basewinxp-v0
>>> is NOT in blockcomputers table
>>> 971 2010-04-01
>>> 15:45:08|12726|10:10|new|State.pm:reservation_failed(346)|exiting 1
>>> 972 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(905)|destructor
>>> called, ref($self)=VCL::new
>>> 973 2010-04-01
>> 15:45:08|12726|10:10|new|utils.pm:delete_computerloadlog_reservation(7551)|removing
>>> computerloadlog entries matching loadstate = begin
>>> 974 2010-04-01
>> 15:45:08|12726|10:10|new|utils.pm:delete_computerloadlog_reservation(7598)|deleted
>>> rows from computerloadlog for reservation id=10
>>> 975 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(912)|removed
>>> computerloadlog rows with loadstate=begin for reservation
>>> 976 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(924)|number of
>>> database handles state process created: 1
>>> 977 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(933)|process
>>> has a database handle stored in $ENV{dbh}, attempting disconnect
>>> 978 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(935)|$ENV{dbh}:
>>> database disconnect successful
>>> 979 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(949)|VCL::new
>>> process 12726 exiting
>>> 980 2010-04-01 15:45:08|5754|vcld:REAPER(744)|VCL process exited for
>>> reservation 10
>>> 981 2010-04-01 15:45:08|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:45:08
>>> 982 2010-04-01 15:45:13|5754|vcld:main(165)|lastcheckin time updated for
>>> management node 1: 2010-04-01 15:45:13
>>>
>>>
>> ================================================================
>> =====================
>>> Thank you,
>>> Amit
>>>
>>>
>>>
--
Andy Kurth
Virtual Computing Lab
Office of Information Technology
North Carolina State University
andy_kurth@ncsu.edu
919.513.4090
Re: VCL Getting Reservation to work
Posted by Ryan Johnson <rj...@gwmail.gwu.edu>.
You can use the full, free license of ESXi, but you just have to make sure
you do not update past update 3. In update 4 Vmware disabled remote
commands which need write permissions i.e. registering VM's, powering them
on/off. As for ssh, just make sure that when you try it manually you use
the same command that VCL is using and that you do not get prompted for
anything, i.e. password input or to accept a new host key. It might take a
long time for ssh to start or for ssh to respond, but I believe VCL will try
for something like 800 seconds.
On Tue, Apr 13, 2010 at 1:03 PM, Kumar, Amit H. <AH...@odu.edu> wrote:
> Hi Ryan,
>
> Yes my ESXi server was ahead of all other computers that were available,
> Once I changed the state of the ESXi server to inuse. The next available
> VMguest computer was picked up and the process continued and worked.
>
> Also we noticed that the license we had put in for the ESXi disabled remote
> controlling of the VMguest using the perl toolkit API/vshpere;
> Interestingly when we changed back to the evaluation version the VMgues was
> registering on the VMhost correctly and was now able to power on as well.
>
> Now at the last mile Stone: Though I can directly ssh, to my vmguest that
> is powered on, from management node: The reservation process fails because
> it complains that it waiting enough time to ssh to the Vmguest and couldn't.
> I noticed that when I tried to SSH manually from the management host to the
> vmgues powered on during the reservation process, there was a significant
> delay in ssh response, I would say about 20-30secs. Not sure if this could
> cause the problem or not. I looked at the run_ssh_command code, I see it
> progressively increases the delay so not sure what is missing here ?????
>
> Hope any of you can shed some more help on me ...
>
> Last few lined of the log:
> <snip>
> No recipient addresses found in header
> 2010-04-13 12:39:38|13024|23:23|new|utils.pm:mail(1348)|SUCCESS -- Sending
> mail To: , PROBLEM -- esx.pm
>
> |13024|23:23|new| ---- CRITICAL ----
> |13024|23:23|new| 2010-04-13 12:39:37|13024|23:23|new|esx.pm:load(521)|waited
> acceptable amount of time for sshd to become active, please check
> esx3-basewinxp-v0 on esx35-hypervisor-1
> |13024|23:23|new| ( 0) utils.pm, notify (line: 737)
> |13024|23:23|new| (-1) esx.pm, load (line: 521)
> |13024|23:23|new| (-2) new.pm, reload_image (line: 665)
> |13024|23:23|new| (-3) new.pm, process (line: 266)
> |13024|23:23|new| (-4) vcld, make_new_child (line: 594)
> |13024|23:23|new| (-5) vcld, main (line: 341)
>
>
> |13024|23:23|new| ---- WARNING ----
> |13024|23:23|new| 2010-04-13 12:39:38|13024|23:23|new|new.pm:reload_image(670)|esx3-basewinxp-v0
> failed to load on esx3-basewinxp-v0, returning
> |13024|23:23|new| ( 0) utils.pm, notify (line: 737)
> |13024|23:23|new| (-1) new.pm, reload_image (line: 670)
> |13024|23:23|new| (-2) new.pm, process (line: 266)
> |13024|23:23|new| (-3) vcld, make_new_child (line: 594)
> |13024|23:23|new| (-4) vcld, main (line: 341)
>
> 2010-04-13 12:39:38|13024|23:23|new|utils.pm:insertloadlog(4710)|inserted
> computer=3, loadimagefailed, esx3-basewinxp-v0 failed to load on
> esx3-basewinxp-v0
>
> |13024|23:23|new| ---- WARNING ----
> |13024|23:23|new| 2010-04-13 12:39:38|13024|23:23|new|new.pm:process(313)|failed
> to load esx3-basewinxp-v0 with esx3-basewinxp-v0
> |13024|23:23|new| ( 0) utils.pm, notify (line: 737)
> |13024|23:23|new| (-1) new.pm, process (line: 313)
> |13024|23:23|new| (-2) vcld, make_new_child (line: 594)
> |13024|23:23|new| (-3) vcld, main (line: 341)
>
> 2010-04-13
> 12:39:38|13024|23:23|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> to retrieve current state of computer esx3-basewinxp-v0 from the database
> 2010-04-13
> 12:39:38|13024|23:23|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> current state of computer esx3-basewinxp-v0 from the database: reloading
> 2010-04-13 12:39:38|13024|23:23|new|DataStructure.pm:_automethod(697)|data
> structure updated:
> $self->request_data->{reservation}{23}{computer}{state}{name}
> |13024|23:23|new| computer_state_name = reloading
> No recipient addresses found in header
> 2010-04-13 12:39:38|13024|23:23|new|utils.pm:mail(1348)|SUCCESS -- Sending
> mail To: , PROBLEM -- State.pm
>
> |13024|23:23|new| ---- CRITICAL ----
> |13024|23:23|new| 2010-04-13
> 12:39:38|13024|23:23|new|State.pm:reservation_failed(290)|reservation failed
> on esx3-basewinxp-v0: process failed after trying to load or make available
> |13024|23:23|new| ( 0) utils.pm, notify (line: 737)
> |13024|23:23|new| (-1) State.pm, reservation_failed (line: 290)
> |13024|23:23|new| (-2) new.pm, process (line: 316)
> |13024|23:23|new| (-3) vcld, make_new_child (line: 594)
> |13024|23:23|new| (-4) vcld, main (line: 341)
>
> 2010-04-13 12:39:38|13024|23:23|new|utils.pm:insertloadlog(4710)|inserted
> computer=3, failed, process failed after trying to load or make available
> 2010-04-13
> 12:39:38|13024|23:23|new|State.pm:reservation_failed(293)|inserted
> computerloadlog entry
> 2010-04-13
> 12:39:38|13024|23:23|new|State.pm:reservation_failed(301)|updated log ending
> value to 'failed', logid=22
> 2010-04-13 12:39:38|13024|23:23|new|utils.pm:update_computer_state(2228)|computer
> 3 state updated to: failed
> 2010-04-13
> 12:39:38|13024|23:23|new|State.pm:reservation_failed(312)|computer
> esx3-basewinxp-v0 (3) state set to failed
> 2010-04-13 12:39:38|13024|23:23|new|utils.pm:update_request_state(2186)|request
> 23 state updated to: failed, laststate to: new
> 2010-04-13 12:39:38|13024|23:23|new|State.pm:reservation_failed(325)|set
> request state to 'failed'/'new'
> 2010-04-13 12:39:38|13024|23:23|new|utils.pm:is_inblockrequest(6972)|zero
> rows were returned from database select
> 2010-04-13
> 12:39:38|13024|23:23|new|State.pm:reservation_failed(343)|esx3-basewinxp-v0
> is NOT in blockcomputers table
> 2010-04-13
> 12:39:38|13024|23:23|new|State.pm:reservation_failed(346)|exiting 1
> 2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(905)|destructor
> called, ref($self)=VCL::new
> 2010-04-13 12:39:38|13024|23:23|new|utils.pm:delete_computerloadlog_reservation(7551)|removing
> computerloadlog entries matching loadstate = begin
> 2010-04-13 12:39:38|13024|23:23|new|utils.pm:delete_computerloadlog_reservation(7598)|deleted
> rows from computerloadlog for reservation id=23
> 2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(912)|removed
> computerloadlog rows with loadstate=begin for reservation
> 2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(924)|number of
> database handles state process created: 1
> 2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(933)|process has a
> database handle stored in $ENV{dbh}, attempting disconnect
> 2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(935)|$ENV{dbh}:
> database disconnect successful
> 2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(949)|VCL::new process
> 13024 exiting
> </snip>
>
>
>
> Thank you,
> Amit
>
> From: Ryan Johnson [mailto:rjohn06@gwmail.gwu.edu]
> Sent: Monday, April 05, 2010 10:27 AM
> To: Kumar, Amit H.
> Subject: Re: VCL Getting Reservation to work
>
> So for every VM you want to use you have to have a row for it in your
> computer table. If you are doing manual reservations then you should set
> the computerid column of the manual reservation you are creating to the id
> of a VM in the computer table. I think you were passing the id of the ESXi
> server as the id in the reservation for your test.
> On Mon, Apr 5, 2010 at 10:13 AM, Kumar, Amit H. <AHKumar@odu.edu<mailto:
> AHKumar@odu.edu>> wrote:
> Hi Ryan,
> I don't have the reservation table for that trial.
> Yes I am doing manual reservation. Not sure what do you mean by "specify a
> vm as the computer id in your reservation row" Could you please elaborate on
> that.
> I do have my vm's assigned to my Vmhosts in the Vmprofile tab. Not sure
> what additional things we have to do here?
>
> Thank you,
> Amit
>
> From: Ryan Johnson [mailto:rjohn06@gwmail.gwu.edu<mailto:
> rjohn06@gwmail.gwu.edu>]
> Sent: Friday, April 02, 2010 3:01 PM
> To: Kumar, Amit H.
> Subject: Re: [Spam: 6.0] Re: [Spam: 6.0] Re: VCL Getting Reservation to
> work
>
> I think the problem might have something to do with the computer you
> specified for the reservation. After looking at the logs, I see that it
> loads an OS module and then tries to run an ssh command on your hypervisor.
> If you are using manual reservations, then you should specify a vm as the
> computer id in your reservation row. If you still have the reservation
> table info from this could you attach it?. Eventually, you will have to map
> the vm's in the computer table to the hypervisor you are using.
> On Thu, Apr 1, 2010 at 4:48 PM, Kumar, Amit H. <AHKumar@odu.edu<mailto:
> AHKumar@odu.edu>> wrote:
> Attached are the screen shots.
>
>
> Thank you,
> Amit
>
>
> > -----Original Message-----
> > From: Ryan Johnson [mailto:rjohn06@gwmail.gwu.edu<mailto:
> rjohn06@gwmail.gwu.edu>]
> > Sent: Thursday, April 01, 2010 4:23 PM
> > To: vcl-dev@incubator.apache.org<ma...@incubator.apache.org>
> > Subject: [Spam: 6.0] Re: [Spam: 6.0] Re: VCL Getting Reservation to work
> >
> > Yea, that is a little hard on the eyes, any chance you could send screen
> > shots in a view where the columns are lined up with the column names?
> >
> > On Thu, Apr 1, 2010 at 4:00 PM, Kumar, Amit H. <AHKumar@odu.edu<mailto:
> AHKumar@odu.edu>> wrote:
> >
> > > The Savior! Thank you Ryan getting back!!!
> > >
> > > Not sure if this is the best format for readability...please let me
> know
> > > otherwise.
> > >
> > > I am also a recent log that I tried with verbose ssh enabled etc, I
> haven't
> > > looked at it yet. It went past my previous log. .....................
> > >
> > > Image: Table: Rows: 10
> > > ====================
> > > id name prettyname ownerid platformid OSid imagemetaid minram
> > > minprocnumber minprocspeed minnetwork maxconcurrent reloadtime
> deleted
> > > test lastupdate forcheckout maxinitialtime project size
> architecture
> > > description usage basedoffrevisionid
> > > 1 winxp-base1-v0 No Apps (WinXP) 1 [->admin] 1 [->i386] 7 [->winxp]
> NULL 0
> > > 1 0 10 NULL 14 0 0 2007-04-11 16:07:38 1 0 vcl 1045 x86 NULL NULL 0
> > > 2 rhel4-base2-v0 Red Hat Enterprise Linux 4.4 Base (KS) 1 [->admin] 1
> > > [->i386] 12 [->rhel4] NULL 1024 1 1024 100 NULL 13 0 0 2007-03-02
> 16:33:33 1
> > > 0 vcl 0 x86 NULL NULL 0
> > > 3 rh4image-base3-v0 RHEL4 base (image) 1 [->admin] 1 [->i386] 13
> > > [->rh4image] NULL 10 1 1024 100 NULL 10 0 0 2007-01-24 15:02:07 1 0 vcl
> 1450
> > > x86 NULL NULL 0
> > > 4 noimage No Image 1 [->admin] 1 [->i386] 2 [->sun4x_58] NULL 0 1 0 10
> NULL
> > > 0 0 0 NULL 1 0 vcl 1450 x86 NULL NULL 0
> > > 5 rhfc5-fc5base5-v0 Red Hat Fedora Core 5 base (KS) 1 [->admin] 1
> [->i386]
> > > 15 [->rhfc5] NULL 1024 1 1024 100 NULL 13 0 0 2006-10-02 10:04:24 1 0
> vcl
> > > 1450 x86 NULL NULL 0
> > > 6 fc5image-FC5baseRHFC56-v0 FC5base(RHFC5) 1 [->admin] 1 [->i386] 14
> > > [->fc5image] NULL 64 1 500 10 NULL 10 0 0 2006-10-02 14:13:26 1 0 vcl 0
> x86
> > > NULL NULL 0
> > > 7 vmwarewinxp-base7-v1 No Apps (WinXP vmware) 1 [->admin] 1 [->i386] 16
> > > [->vmwarewinxp] NULL 512 1 1024 100 NULL 5 0 0 2007-04-04 09:45:38 1 0
> vcl
> > > 3244 x86 NULL NULL 0
> > > 8 rh4image-VMwareserverhostRHEL48-v3 VMware server host (RHEL4) 1
> [->admin]
> > > 1 [->i386] 13 [->rh4image] NULL 64 1 500 10 NULL 10 0 0 2007-11-26
> 11:23:53
> > > 1 0 vcl 1450 x86 NULL NULL 0
> > > 9 esx35-base-v0 VMware ESX 3.5 standard server 1 [->admin] 1 [->i386]
> 20
> > > [->esx35] NULL 2048 2 2000 100 NULL 9 0 0 2008-03-24 14:23:54 1 0 vcl
> 1450
> > > x86 NULL NULL 0
> > > 10 esx3-basewinxp-v0 No Apps (ESXi WinXP) 1 [->admin] 1 [->i386] 20
> > > [->esx35] NULL 512 1 1024 100 NULL 9 0 0 2010-03-24 17:35:12 1 0 vcl
> 4100
> > > x86 NULL NULL 0
> > > /End Image Table====================
> > >
> > >
> > > ImageRevision Table: Rows: 11
> > > ===========================
> > > id imageid revision userid datecreated deleted datedeleted
> > > production comments imagename
> > > 1 1 [->No Apps (WinXP)] 0 1 [->admin] 2007-03-01 14:46:26 0 NULL 1 NULL
> > > winxp-base1-v0
> > > 2 2 [->Red Hat Enterprise Linux 4.4 Base (KS)] 0 1 [->admin] 2006-09-22
> > > 16:33:24 0 NULL 1 NULL rhel4-base2-v0
> > > 3 3 [->RHEL4 base (image)] 0 1 [->admin] 2007-01-24 15:04:11 0 NULL 1
> NULL
> > > rh4image-base3-v0
> > > 4 4 [->No Image] 0 1 [->admin] 1980-01-01 00:00:00 0 NULL 1 NULL
> noimage
> > > 5 5 [->Red Hat Fedora Core 5 base (KS)] 0 1 [->admin] 2006-10-02
> 10:04:24 0
> > > NULL 1 NULL rhfc5-fc5base5-v0
> > > 6 6 [->FC5base(RHFC5)] 0 1 [->admin] 2006-10-02 14:13:26 0 NULL 1 NULL
> > > fc5image-FC5baseRHFC56-v0
> > > 7 7 [->No Apps (WinXP vmware)] 0 1 [->admin] 2007-04-03 11:24:12 0 NULL
> 1
> > > NULL vmwarewinxp-base7-v0
> > > 8 8 [->VMware server host (RHEL4)] 0 1 [->admin] 2007-01-30 10:50:56 0
> NULL
> > > 1 NULL rh4image-VMwareserverhostRHEL48-v0
> > > 9 9 [->VMware ESX 3.5 standard server] 0 1 [->admin] 2008-03-24
> 14:23:54 0
> > > NULL 1 NULL esx35-base-v0
> > > 10 7 [->No Apps (WinXP vmware)] 1 1 [->admin] 2010-03-18 14:56:50 0
> NULL 1
> > > NULL vmwarewinxp-base7-v1
> > > 11 10 [->No Apps (ESXi WinXP)] 0 1 [->admin] 2010-03-24 17:35:12 0 NULL
> 1
> > > NULL esx3-basewinxp-v0
> > > /End ImageRevision Table====================
> > >
> > >
> > > Computer Table: Rows: 3
> > > ============================================
> > > id stateid ownerid platformid scheduleid currentimageid
> > > preferredimageid nextimageid imagerevisionid RAM procnumber
> procspeed
> > > network hostname IPaddress privateIPaddress eth0macaddress
> > > eth1macaddress type provisioningid drivetype deleted notes
> lastcheck
> > > location dsa dsapub rsa rsapub host hostpub vmhostid vmtypeid
> > > 1 10 [->maintenance] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No
> Image] 0
> > > 0 0 512 2 2200 1000 resxi-2.odu.edu<http://resxi-2.odu.edu>
> 128.82.128.218 NULL NULL NULL
> > > virtualmachine 4 [->vmware_esx] hda 1 NULL NULL NULL NULL NULL NULL
> > NULL
> > > [BLOB - 0B] NULL NULL NULL
> > > 2 2 [->available] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image]
> 0 0
> > > 0 20480 8 2200 1000 esx35-hypervisor-1.odu.edu<
> http://esx35-hypervisor-1.odu.edu> 128.82.128.252 NULL NULL
> > > NULL blade 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL [BLOB
> > -
> > > 0B] NULL NULL NULL
> > > 3 5 [->failed] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0
> 0
> > > 512 2 2200 100 esx3-basewinxp-v0.odu.edu<
> http://esx3-basewinxp-v0.odu.edu> 128.82.128.217 NULL NULL NULL
> > > virtualmachine 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL
> > [BLOB
> > > - 0B] NULL 1 NULL
> > > /End Computer Table======================================
> > >
> > >
> > > Vmprofile Table: Rows: 4
> > > ========================
> > > id profilename vmtypeid imageid nasshare datastorepath vmpath
> > > virtualswitch0 virtualswitch1 vmdisk username password
> > > 1 VMware GSX standard 3 [->vmwareGSX] 8 [->VMware server host (RHEL4)]
> NULL
> > > /var/lib/vmware/Virtual Machines NULL VMnet0 VMnet2 localdisk NULL NULL
> > > 2 Vmware ESX standard network mounted share 5 [->vmwareESX3] 9
> [->VMware
> > > ESX 3.5 standard server] NULL /vmfs/volumes/nfs1 /vmfs/volumes/storage1
> VM
> > > Network Virtual Machine Public Network networkdisk NULL NULL
> > > 3 Vmware ESX standard localdisk 6 [->vmwareESXi] 9 [->VMware ESX 3.5
> > > standard server] NULL 192.168.129.37:/install/image/vmware_images
> > > 192.168.129.37:/install/image/vmware_images VM Network Private Vlan328
> > > localdisk root
> > > 4 Vmware ESX SAN 5 [->vmwareESX3] 9 [->VMware ESX 3.5 standard server]
> > NULL
> > > /vmfs/volumes/NetApp /vmfs/volumes/storage1 Intranet2 MCNC Public
> > > networkdisk NULL NULL
> > > /End Vmprofile===============================
> > >
> > > VMhost Table: Rows: 1
> > > =====================
> > > id computerid vmlimit vmprofileid vmkernalnic vmwaredisk
> > > 1 2 [->esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu>]
> 2 3 NULL localdisk
> > > /End Vmhost Table==========================
> > >
> > >
> > > ================================= LOG FILE
> > > ===========================================
> > > 13 2010-04-01 15:37:19|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:37:19
> > > 14 2010-04-01 15:37:24|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:37:24
> > > 15 2010-04-01
> > 15:37:24|5754|9:9|deleted|utils.pm:
> reservation_being_processed(9634)|computerloadlog
> > > 'begin' entry does NOT exist for reservation 9
> > > 16 2010-04-01
> > 15:37:24|5754|9:9|deleted|utils.pm:
> reservation_being_processed(9686)|reservation
> > > is NOT currently being processed
> > > 17 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(276)|reservation
> 9 is
> > > NOT already being processed
> > > 18
> > > 19 |5754|9:9|deleted| ---- WARNING ----
> > > 20 |5754|9:9|deleted| 2010-04-01
> > 15:37:24|5754|9:9|deleted|utils.pm:
> get_request_info(5218)|preferredimageid
> > > is not set for computer id=2
> > > 21 |5754|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 22 |5754|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> get_request_info (line: 5218)
> > > 23 |5754|9:9|deleted| (-2) vcld, main (line: 280)
> > > 24
> > > 25 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:
> get_request_info(5354)|standalone
> > > affiliation found: Local
> > > 26 2010-04-01
> > 15:37:24|5754|9:9|deleted|utils.pm:
> get_management_node_info(6815)|management
> > > node info retrieved from database for colbert
> > > 27 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(281)|retrieved
> > > request information from database
> > > 28 2010-04-01
> > >
> 15:37:24|5754|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > > true: parent reservation ID for this request: 9
> > > 29 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:
> update_request_state(2186)|request
> > > 9 state updated to: pending, laststate to: deleted
> > > 30 2010-04-01
> > 15:37:24|5754|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > > entry not inserted, reservation has been deleted
> > > 31 2010-04-01
> > > 15:37:24|5754|9:9|deleted|vcld:make_new_child(489)|request will be
> processed
> > > by reclaim.pm<http://reclaim.pm>
> > > 32 2010-04-01
> 15:37:24|5754|9:9|deleted|vcld:make_new_child(509)|loaded
> > > VCL::reclaim module
> > > 33 2010-04-01
> 15:37:24|12719|9:9|deleted|vcld:make_new_child(582)|vcld
> > > environment variable set to 0 for this process
> > > 34 2010-04-01
> 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> > > called, class=VCL::reclaim
> > > 35 2010-04-01
> > > 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::reclaim object
> created
> > > 36 2010-04-01
> > > 15:37:24|5754|9:9|deleted|vcld:make_new_child(566)|current number of
> forked
> > > kids: 1
> > > 37 2010-04-01
> > > 15:37:24|12719|9:9|deleted|State.pm:initialize(85)|obtained a database
> > > handle for this state process, stored as $ENV{dbh}
> > > 38 2010-04-01
> > > 15:37:24|12719|9:9|deleted|State.pm:check_image_os(820)|no corrections
> need
> > > to be made, not an imaging request, returning 1
> > > 39 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > > true: parent reservation ID for this request: 9
> > > 40 2010-04-01
> > 15:37:24|12719|9:9|deleted|utils.pm:
> rename_vcld_process(7863)|reservation
> > > count: 1
> > > 41 2010-04-01
> > 15:37:24|12719|9:9|deleted|utils.pm:
> rename_vcld_process(7882)|PARENTIMAGE:
> > > 1
> > > 42 2010-04-01
> > 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7883)|SUBIMAGE:
> > > 0
> > > 43 2010-04-01
> > 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7901)|renamed
> > > process to 'vcld VCL::reclaim 9:9 deleted'
> > > 44 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > > true: parent reservation ID for this request: 9
> > > 45 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > > true: parent reservation ID for this request: 9
> > > 46 2010-04-01
> > > 15:37:24|12719|9:9|deleted|State.pm:initialize(121)|attempting to load
> > > provisioning module: VCL::Module::Provisioning::esx
> > > 47 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:
> initialize(138)|esx
> > > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > > 48 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:
> initialize(140)|vmware
> > > ESX module initialized
> > > 49 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|State.pm:initialize(127)|VCL::Module::Provisioning::esx
> > > module loaded
> > > 50 2010-04-01
> 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> > > called, class=VCL::Module::Provisioning::esx
> > > 51 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::Provisioning::esx
> > > object created
> > > 52 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:
> initialize(138)|esx
> > > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > > 53 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:
> initialize(140)|vmware
> > > ESX module initialized
> > > 54 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|State.pm:initialize(131)|VCL::Module::Provisioning::esx
> > > provisioner object created
> > > 55 2010-04-01
> > > 15:37:24|12719|9:9|deleted|State.pm:initialize(145)|attempting to load
> OS
> > > module: VCL::Module::OS::Linux
> > > 56 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|State.pm:initialize(151)|VCL::Module::OS::Linux
> > > module loaded
> > > 57 2010-04-01
> 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> > > called, class=VCL::Module::OS::Linux
> > > 58 2010-04-01
> > > 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::OS::Linux
> object
> > > created
> > > 59 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|State.pm:initialize(154)|VCL::Module::OS::Linux
> > > OS object created
> > > 60 2010-04-01
> > > 15:37:24|12719|9:9|deleted|State.pm:initialize(169)|returning 1
> > > 61 2010-04-01
> > > 15:37:24|12719|9:9|deleted|vcld:make_new_child(591)|VCL::reclaim object
> > > created and initialized
> > > 62 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation
> > > remote IP is not defined
> > > 63 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1946)|attempting
> > > to retrieve current state of computer esx35-hypervisor-1 from the
> database
> > > 64 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > > current state of computer esx35-hypervisor-1 from the database:
> available
> > > 65 2010-04-01
> > > 15:37:24|12719|9:9|deleted|DataStructure.pm:_automethod(697)|data
> structure
> > > updated: $self->request_data->{reservation}{9}{computer}{state}{name}
> > > 66 |12719|9:9|deleted| computer_state_name = available
> > > 67 2010-04-01
> > 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > > entry not inserted, reservation has been deleted
> > > 68 2010-04-01
> > 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > > entry not inserted, reservation has been deleted
> > > 69 2010-04-01
> > 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > > entry not inserted, reservation has been deleted
> > > 70 2010-04-01
> > 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > > entry not inserted, reservation has been deleted
> > > 71 2010-04-01
> > 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > > entry not inserted, reservation has been deleted
> > > 72 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:
> process(138)|computer
> > > type is blade
> > > 73 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:
> process(142)|request
> > > laststate is reserved, attempting to sanitize computer
> > > 74 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:
> process(172)|calling
> > > VCL::Module::OS::Linux::sanitize() subroutine
> > > 75 2010-04-01
> > >
> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation
> > > remote IP is not defined
> > > 76 2010-04-01
> > 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on esx35-hypervisor-1:
> > > 77 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'netstat -an' 2>&1
> > > 78
> > > 79 |12719|9:9|deleted| ---- WARNING ----
> > > 80 |12719|9:9|deleted| 2010-04-01
> > 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 1/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an,
> exit
> > > status: 255, SSH exits with the exit status of the remote command or
> with
> > > 255 if an error occurred, output:
> > > 81 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat:
> not
> > > found
> > > 82 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 83 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 84 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > > 85 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > > 86 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 87 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 88 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 89
> > > 90 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 2 seconds before making next SSH attempt
> > > 91 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 2/3: executing SSH command on esx35-hypervisor-1:
> > > 92 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'netstat -an' 2>&1
> > > 93
> > > 94 |12719|9:9|deleted| ---- WARNING ----
> > > 95 |12719|9:9|deleted| 2010-04-01
> > 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 2/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an,
> exit
> > > status: 255, SSH exits with the exit status of the remote command or
> with
> > > 255 if an error occurred, output:
> > > 96 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat:
> not
> > > found
> > > 97 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 98 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 99 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > > 100 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > > 101 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 102 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 103 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 104
> > > 105 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 15 seconds before making next SSH attempt
> > > 106 2010-04-01 15:37:29|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:37:29
> > > 107 2010-04-01 15:37:34|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:37:34
> > > 108 2010-04-01 15:37:39|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:37:39
> > > 109 2010-04-01
> > 15:37:39|5754|10:10|new|utils.pm:
> reservation_being_processed(9634)|computerloadlog
> > > 'begin' entry does NOT exist for reservation 10
> > > 110 2010-04-01
> > 15:37:39|5754|10:10|new|utils.pm:
> reservation_being_processed(9686)|reservation
> > > is NOT currently being processed
> > > 111 2010-04-01 15:37:39|5754|10:10|new|vcld:main(276)|reservation 10
> is
> > > NOT already being processed
> > > 112
> > > 113 |5754|10:10|new| ---- WARNING ----
> > > 114 |5754|10:10|new| 2010-04-01
> > 15:37:39|5754|10:10|new|utils.pm:get_request_info(5218)|preferredimageid
> > > is not set for computer id=3
> > > 115 |5754|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 116 |5754|10:10|new| (-1) utils.pm<http://utils.pm>,
> get_request_info (line: 5218)
> > > 117 |5754|10:10|new| (-2) vcld, main (line: 280)
> > > 118
> > > 119 2010-04-01 15:37:39|5754|10:10|new|utils.pm:
> get_request_info(5354)|standalone
> > > affiliation found: Local
> > > 120 2010-04-01
> > 15:37:39|5754|10:10|new|utils.pm:
> get_management_node_info(6815)|management
> > > node info retrieved from database for colbert
> > > 121 2010-04-01 15:37:39|5754|10:10|new|vcld:main(281)|retrieved
> request
> > > information from database
> > > 122 2010-04-01
> > >
> 15:37:39|5754|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > > true: parent reservation ID for this request: 10
> > > 123 2010-04-01 15:37:39|5754|10:10|new|utils.pm:
> update_request_state(2186)|request
> > > 10 state updated to: pending, laststate to: new
> > > 124 2010-04-01 15:37:39|5754|10:10|new|utils.pm:
> insertloadlog(4710)|inserted
> > > computer=3, begin, beginning to process, state is new
> > > 125 2010-04-01
> 15:37:39|5754|10:10|new|vcld:make_new_child(509)|loaded
> > > VCL::new module
> > > 126 2010-04-01
> 15:37:39|12726|10:10|new|vcld:make_new_child(582)|vcld
> > > environment variable set to 0 for this process
> > > 127 2010-04-01
> 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> > > called, class=VCL::new
> > > 128 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::new
> > > object created
> > > 129 2010-04-01
> 15:37:39|5754|10:10|new|vcld:make_new_child(566)|current
> > > number of forked kids: 2
> > > 130 2010-04-01
> 15:37:39|12726|10:10|new|State.pm:initialize(85)|obtained
> > > a database handle for this state process, stored as $ENV{dbh}
> > > 131 2010-04-01
> 15:37:39|12726|10:10|new|State.pm:check_image_os(820)|no
> > > corrections need to be made, not an imaging request, returning 1
> > > 132 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > > true: parent reservation ID for this request: 10
> > > 133 2010-04-01
> > 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7863)|reservation
> > > count: 1
> > > 134 2010-04-01
> > 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7882)|PARENTIMAGE:
> > > 1
> > > 135 2010-04-01
> > 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7883)|SUBIMAGE:
> > > 0
> > > 136 2010-04-01
> > 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7901)|renamed
> > > process to 'vcld VCL::new 10:10 new'
> > > 137 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > > true: parent reservation ID for this request: 10
> > > 138 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > > true: parent reservation ID for this request: 10
> > > 139 2010-04-01
> > > 15:37:39|12726|10:10|new|State.pm:initialize(121)|attempting to load
> > > provisioning module: VCL::Module::Provisioning::esx
> > > 140 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx
> > > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > > 141 2010-04-01 15:37:39|12726|10:10|new|esx.pm:
> initialize(140)|vmware
> > > ESX module initialized
> > > 142 2010-04-01
> > >
> 15:37:39|12726|10:10|new|State.pm:initialize(127)|VCL::Module::Provisioning::esx
> > > module loaded
> > > 143 2010-04-01
> 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> > > called, class=VCL::Module::Provisioning::esx
> > > 144 2010-04-01
> > >
> 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::Provisioning::esx
> > > object created
> > > 145 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx
> > > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > > 146 2010-04-01 15:37:39|12726|10:10|new|esx.pm:
> initialize(140)|vmware
> > > ESX module initialized
> > > 147 2010-04-01
> > >
> 15:37:39|12726|10:10|new|State.pm:initialize(131)|VCL::Module::Provisioning::esx
> > > provisioner object created
> > > 148 2010-04-01
> > > 15:37:39|12726|10:10|new|State.pm:initialize(145)|attempting to load OS
> > > module: VCL::Module::OS::Linux
> > > 149 2010-04-01
> > >
> 15:37:39|12726|10:10|new|State.pm:initialize(151)|VCL::Module::OS::Linux
> > > module loaded
> > > 150 2010-04-01
> 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> > > called, class=VCL::Module::OS::Linux
> > > 151 2010-04-01
> > > 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::OS::Linux
> object
> > > created
> > > 152 2010-04-01
> > >
> 15:37:39|12726|10:10|new|State.pm:initialize(154)|VCL::Module::OS::Linux OS
> > > object created
> > > 153 2010-04-01
> > > 15:37:39|12726|10:10|new|State.pm:initialize(169)|returning 1
> > > 154 2010-04-01
> > > 15:37:39|12726|10:10|new|vcld:make_new_child(591)|VCL::new object
> created
> > > and initialized
> > > 155 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > > true: parent reservation ID for this request: 10
> > > 156 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > > to retrieve current state of computer esx3-basewinxp-v0 from the
> database
> > > 157 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > > current state of computer esx3-basewinxp-v0 from the database:
> available
> > > 158 2010-04-01
> > > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data
> structure
> > > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > > 159 |12726|10:10|new| computer_state_name = available
> > > 160
> > > 161 |12726|10:10|new| ---- WARNING ----
> > > 162 |12726|10:10|new| 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > > data has not been initialized for get_computer_preferredimage_id:
> > > $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> > > 163 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 164 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > > 165 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > > 166 |12726|10:10|new| (-3) new.pm<http://new.pm>, process (line:
> 106)
> > > 167 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> > > 168 |12726|10:10|new| (-5) vcld, main (line: 341)
> > > 169
> > > 170
> > > 171 |12726|10:10|new| ---- WARNING ----
> > > 172 |12726|10:10|new| 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > > data has not been initialized for get_computer_preferredimage_name:
> > > $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> > > 173 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 174 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > > 175 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > > 176 |12726|10:10|new| (-3) new.pm<http://new.pm>, process (line:
> 107)
> > > 177 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> > > 178 |12726|10:10|new| (-5) vcld, main (line: 341)
> > > 179
> > > 180 2010-04-01
> > 15:37:39|12726|10:10|new|utils.pm:
> get_management_node_id(9364)|environment
> > > variable: 1
> > > 181 2010-04-01 15:37:39|12726|10:10|new|new.pm:
> process(134)|reservation
> > > is parent = 1
> > > 182 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(135)|preload
> > > only = 0
> > > 183 2010-04-01 15:37:39|12726|10:10|new|new.pm:
> process(136)|originating
> > > request state = new
> > > 184 2010-04-01 15:37:39|12726|10:10|new|new.pm:
> process(137)|originating
> > > request laststate = new
> > > 185 2010-04-01 15:37:39|12726|10:10|new|new.pm:
> process(138)|originating
> > > computer state = available
> > > 186 2010-04-01 15:37:39|12726|10:10|new|new.pm:
> process(139)|originating
> > > computer type = virtualmachine
> > > 187 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > > true: parent reservation ID for this request: 10
> > > 188 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > > to retrieve current state of computer esx3-basewinxp-v0 from the
> database
> > > 189 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > > current state of computer esx3-basewinxp-v0 from the database:
> available
> > > 190 2010-04-01
> > > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data
> structure
> > > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > > 191 |12726|10:10|new| computer_state_name = available
> > > 192
> > > 193 |12726|10:10|new| ---- WARNING ----
> > > 194 |12726|10:10|new| 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > > data has not been initialized for get_computer_preferredimage_id:
> > > $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> > > 195 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 196 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > > 197 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > > 198 |12726|10:10|new| (-3) new.pm<http://new.pm>,
> computer_not_being_used (line: 750)
> > > 199 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line:
> 170)
> > > 200 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > > 201 |12726|10:10|new| (-6) vcld, main (line: 341)
> > > 202
> > > 203
> > > 204 |12726|10:10|new| ---- WARNING ----
> > > 205 |12726|10:10|new| 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > > data has not been initialized for get_computer_preferredimage_name:
> > > $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> > > 206 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 207 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > > 208 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > > 209 |12726|10:10|new| (-3) new.pm<http://new.pm>,
> computer_not_being_used (line: 751)
> > > 210 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line:
> 170)
> > > 211 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > > 212 |12726|10:10|new| (-6) vcld, main (line: 341)
> > > 213
> > > 214 2010-04-01
> > 15:37:39|12726|10:10|new|utils.pm:
> get_management_node_id(9364)|environment
> > > variable: 1
> > > 215 2010-04-01
> > 15:37:39|12726|10:10|new|new.pm:
> computer_not_being_used(788)|esx3-basewinxp-v0
> > > state is available
> > > 216 2010-04-01
> > 15:37:39|12726|10:10|new|new.pm:
> computer_not_being_used(798)|esx3-basewinxp-v0
> > > is available, its state is available
> > > 217 2010-04-01 15:37:39|12726|10:10|new|new.pm:
> process(170)|esx3-basewinxp-v0
> > > is not being used
> > > 218 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > > true: parent reservation ID for this request: 10
> > > 219 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > > to retrieve current state of computer esx3-basewinxp-v0 from the
> database
> > > 220 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > > current state of computer esx3-basewinxp-v0 from the database:
> available
> > > 221 2010-04-01
> > > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data
> structure
> > > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > > 222 |12726|10:10|new| computer_state_name = available
> > > 223
> > > 224 |12726|10:10|new| ---- WARNING ----
> > > 225 |12726|10:10|new| 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > > data has not been initialized for get_computer_preferredimage_id:
> > > $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> > > 226 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 227 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > > 228 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > > 229 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image
> (line: 480)
> > > 230 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line:
> 266)
> > > 231 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > > 232 |12726|10:10|new| (-6) vcld, main (line: 341)
> > > 233
> > > 234
> > > 235 |12726|10:10|new| ---- WARNING ----
> > > 236 |12726|10:10|new| 2010-04-01
> > >
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > > data has not been initialized for get_computer_preferredimage_name:
> > > $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> > > 237 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 238 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > > 239 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > > 240 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image
> (line: 481)
> > > 241 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line:
> 266)
> > > 242 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > > 243 |12726|10:10|new| (-6) vcld, main (line: 341)
> > > 244
> > > 245 2010-04-01
> > 15:37:39|12726|10:10|new|utils.pm:
> get_management_node_id(9364)|environment
> > > variable: 1
> > > 246 2010-04-01 15:37:39|12726|10:10|new|new.pm:
> reload_image(514)|calling
> > > VCL::Module::Provisioning::esx->node_status()
> > > 247 2010-04-01 15:37:39|12726|10:10|new|utils.pm:
> insertloadlog(4710)|inserted
> > > computer=3, statuscheck, checking status of node
> > > 248 $VAR1 = bless( {
> > > 249 'data' => bless( do{\(my $o = 1)},
> > > 'VCL::DataStructure' ),
> > > 250 'os' => bless( {
> > > 251 'provisioner' => $VAR1,
> > > 252 'data' => $VAR1->{'data'}
> > > 253 }, 'VCL::Module::OS::Linux' )
> > > 254 }, 'VCL::Module::Provisioning::esx' );
> > > 255 2010-04-01 15:37:39|12726|10:10|new|esx.pm:
> node_status(787)|Entering
> > > node_status, checking status of esx3-basewinxp-v0
> > > 256 2010-04-01
> > 15:37:39|12726|10:10|new|esx.pm:node_status(788)|request_for_imaging:
> > > 0
> > > 257 2010-04-01 15:37:39|12726|10:10|new|esx.pm:
> node_status(789)|requeseted
> > > image name: esx3-basewinxp-v0
> > > 258 2010-04-01 15:37:39|12726|10:10|new|esx.pm:
> node_status(819)|checking
> > > if esx3-basewinxp-v0 is pingable
> > > 259 2010-04-01 15:37:40|12726|10:10|new|esx.pm:
> node_status(825)|esx3-basewinxp-
> > v0
> > > is not pingable (0)
> > > 260 2010-04-01 15:37:40|12726|10:10|new|new.pm:
> reload_image(560)|node_status
> > > returned a scalar: RELOAD
> > > 261 2010-04-01 15:37:40|12726|10:10|new|new.pm:
> reload_image(602)|node
> > > status is RELOAD, esx3-basewinxp-v0 will be reloaded
> > > 262 2010-04-01 15:37:40|12726|10:10|new|utils.pm:
> insertloadlog(4710)|inserted
> > > computer=3, loadimageblade, esx3-basewinxp-v0 must be reloaded with
> > > esx3-basewinxp-v0
> > > 263 2010-04-01 15:37:40|12726|10:10|new|new.pm:
> reload_image(616)|calling
> > > VCL::Module::Provisioning::esx->does_image_exist()
> > > 264 2010-04-01
> > 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on 192.168.129.37<http://192.168.129.37>:
> > > 265 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22 -x
> > > 192.168.129.37 'ls -1 /install/image/vmware_images/golden 2>&1' 2>&1
> > > 266 2010-04-01
> > 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > > output:
> > > 267 |12726|10:10|new| esx3-basewinxp-v0
> > > 268 2010-04-01 15:37:40|12726|10:10|new|utils.pm:
> run_ssh_command(6276)|SSH
> > > command executed on 192.168.129.37, returning (0, "esx3-basewinxp-v0")
> > > 269 2010-04-01 15:37:40|12726|10:10|new|esx.pm:
> does_image_exist(917)|image
> > > esx3-basewinxp-v0 exists
> > > 270 2010-04-01 15:37:40|12726|10:10|new|new.pm:
> reload_image(619)|esx3-
> > basewinxp-v0
> > > exists on this management node
> > > 271 2010-04-01 15:37:40|12726|10:10|new|utils.pm:
> insertloadlog(4710)|inserted
> > > computer=3, doesimageexists, confirmed image exists
> > > 272 2010-04-01
> > 15:37:40|12726|10:10|new|utils.pm:update_computer_state(2228)|computer
> > > 3 state updated to: reloading
> > > 273 2010-04-01 15:37:40|12726|10:10|new|new.pm:
> reload_image(652)|computer
> > > esx3-basewinxp-v0 state set to reloading
> > > 274 2010-04-01 15:37:40|12726|10:10|new|utils.pm:
> insertloadlog(4710)|inserted
> > > computer=3, info, computer state updated to reloading
> > > 275 2010-04-01 15:37:40|12726|10:10|new|new.pm:
> reload_image(663)|calling
> > > VCL::Module::Provisioning::esx->load() subroutine
> > > 276 2010-04-01 15:37:40|12726|10:10|new|utils.pm:
> insertloadlog(4710)|inserted
> > > computer=3, info, calling VCL::Module::Provisioning::esx->load()
> subroutine
> > > 277 2010-04-01 15:37:40|12726|10:10|new|esx.pm<http://esx.pm>:
> > > load(165)|****************************************************
> > > 278
> > > 279 |12726|10:10|new| ---- WARNING ----
> > > 280 |12726|10:10|new| 2010-04-01
> > >
> 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > > data has not been initialized for get_computer_eth0_mac_address:
> > > $self->request_data->{reservation}{10}{computer}{eth0macaddress}
> > > 281 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 282 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > > 283 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > > 284 |12726|10:10|new| (-3) esx.pm<http://esx.pm>, load (line: 181)
> > > 285 |12726|10:10|new| (-4) new.pm<http://new.pm>, reload_image
> (line: 665)
> > > 286 |12726|10:10|new| (-5) new.pm<http://new.pm>, process (line:
> 266)
> > > 287 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
> > > 288
> > > 289
> > > 290 |12726|10:10|new| ---- WARNING ----
> > > 291 |12726|10:10|new| 2010-04-01
> > >
> 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > > data has not been initialized for get_computer_eth1_mac_address:
> > > $self->request_data->{reservation}{10}{computer}{eth1macaddress}
> > > 292 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 293 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > > 294 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > > 295 |12726|10:10|new| (-3) esx.pm<http://esx.pm>, load (line: 182)
> > > 296 |12726|10:10|new| (-4) new.pm<http://new.pm>, reload_image
> (line: 665)
> > > 297 |12726|10:10|new| (-5) new.pm<http://new.pm>, process (line:
> 266)
> > > 298 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
> > > 299
> > > 300 2010-04-01 15:37:40|12726|10:10|new|esx.pm:load(193)|Calling
> > > get_vmware_host_info
> > > 301 No recipient addresses found in header
> > > 302 2010-04-01 15:37:41|12726|10:10|new|utils.pm:mail(1348)|SUCCESS
> --
> > > Sending mail To: , PROBLEM -- esx.pm<http://esx.pm>
> > > 303
> > > 304 |12726|10:10|new| ---- CRITICAL ----
> > > 305 |12726|10:10|new| 2010-04-01
> > 15:37:40|12726|10:10|new|esx.pm:get_vmware_host_info(1012)|AMIT
> > > DEBUGGING: esx35-hypervisor-1.odu.edu<
> http://esx35-hypervisor-1.odu.edu>
> > > 306 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 307 |12726|10:10|new| (-1) esx.pm<http://esx.pm>,
> get_vmware_host_info (line: 1012)
> > > 308 |12726|10:10|new| (-2) esx.pm<http://esx.pm>, load (line: 194)
> > > 309 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image
> (line: 665)
> > > 310 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line:
> 266)
> > > 311 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > > 312 |12726|10:10|new| (-6) vcld, main (line: 341)
> > > 313
> > > 314 2010-04-01 15:37:41|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 3/3: executing SSH command on esx35-hypervisor-1:
> > > 315 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'netstat -an' 2>&1
> > > 316
> > > 317 |12719|9:9|deleted| ---- WARNING ----
> > > 318 |12719|9:9|deleted| 2010-04-01
> > 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 3/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an,
> exit
> > > status: 255, SSH exits with the exit status of the remote command or
> with
> > > 255 if an error occurred, output:
> > > 319 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat:
> not
> > > found
> > > 320 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 321 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 322 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > > 323 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > > 324 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 325 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 326 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 327
> > > 328
> > > 329 |12719|9:9|deleted| ---- WARNING ----
> > > 330 |12719|9:9|deleted| 2010-04-01
> > 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'netstat -an'
> 2>&1,
> > > exit status: 255, output:
> > > 331 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat:
> not
> > > found
> > > 332 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 333 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6285)
> > > 334 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > > 335 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > > 336 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 337 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 338 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 339
> > > 340 2010-04-01
> > 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on esx35-hypervisor-1:
> > > 341 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > > /etc/ssh/external_sshd_config"' 2>&1
> > > 342
> > > 343 |12719|9:9|deleted| ---- WARNING ----
> > > 344 |12719|9:9|deleted| 2010-04-01
> > 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 1/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx
> > > "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255,
> SSH
> > > exits with the exit status of the remote command or with 255 if an
> error
> > > occurred, output:
> > > 345 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not
> > > found
> > > 346 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 347 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 348 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line:
> 913)
> > > 349 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
> > > 350 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
> > > 351 |12719|9:9|deleted| (-5) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 352 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
> > > 353
> > > 354 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 2 seconds before making next SSH attempt
> > > 355 2010-04-01 15:37:43|12726|10:10|new|esx.pm:
> get_vmware_host_info(1019)|host
> > > info output for esx35-hypervisor-1
> > > 356 |12726|10:10|new| Host Information
> > > 357 |12726|10:10|new| Host Name: esx35-hypervisor-1.odu.edu<
> http://esx35-hypervisor-1.odu.edu>
> > > 358 2010-04-01 15:37:43|12726|10:10|new|esx.pm:
> get_vmware_host_info(1025)|found
> > > hostname_value= esx35-hypervisor-1.odu.edu<
> http://esx35-hypervisor-1.odu.edu>
> > > 359 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(197)|Collected
> > > esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu> for
> vmware host name
> > > 360 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(210)|DATASTORE
> IP
> > > is 192.168.129.37 and DATASTORE_SHARE_PATH is
> /install/image/vmware_images
> > > 361 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(211)|Entered
> ESX
> > > module, loading esx3-basewinxp-v0 on esx3-basewinxp-v0 (on
> > > esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu>) for
> reservation 10
> > > 362 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(212)|Datastore:
> > > 192.168.129.37:/install/image/vmware_images
> > > 363 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(223)|VM info
> > > command: /usr/lib/vmware-vcli/apps/vm/vminfo.pl<http://vminfo.pl>
> --server
> > > 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --username root
> --password
> > > 'xxxxxxxxxxx'
> > > 364 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 2/3: executing SSH command on esx35-hypervisor-1:
> > > 365 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > > /etc/ssh/external_sshd_config"' 2>&1
> > > 366 2010-04-01 15:37:44|12726|10:10|new|esx.pm:load(226)|VM info
> > > output: Virtual Machine esx3-basewinxp-v0 not found.
> > > 367 2010-04-01
> > 15:37:44|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on 192.168.129.37<http://192.168.129.37>:
> > > 368 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22 -x
> > > 192.168.129.37 'rm -rf
> /install/image/vmware_images/inuse/esx3-basewinxp-v0'
> > > 2>&1
> > > 369 2010-04-01 15:37:44|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:37:44
> > > 370
> > > 371 |12719|9:9|deleted| ---- WARNING ----
> > > 372 |12719|9:9|deleted| 2010-04-01
> > 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 2/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx
> > > "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255,
> SSH
> > > exits with the exit status of the remote command or with 255 if an
> error
> > > occurred, output:
> > > 373 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not
> > > found
> > > 374 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 375 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 376 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line:
> 913)
> > > 377 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
> > > 378 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
> > > 379 |12719|9:9|deleted| (-5) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 380 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
> > > 381
> > > 382 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 15 seconds before making next SSH attempt
> > > 383 2010-04-01
> > 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > > output:
> > > 384 |12726|10:10|new| none
> > > 385 2010-04-01 15:37:45|12726|10:10|new|utils.pm:
> run_ssh_command(6276)|SSH
> > > command executed on 192.168.129.37, returning (0, "none")
> > > 386 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(262)|Removed
> old vm
> > > folder
> > > 387 2010-04-01
> > 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on 192.168.129.37<http://192.168.129.37>:
> > > 388 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22 -x
> > > 192.168.129.37 'mkdir
> /install/image/vmware_images/inuse/esx3-basewinxp-v0'
> > > 2>&1
> > > 389 2010-04-01
> > 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > > output:
> > > 390 |12726|10:10|new| none
> > > 391 2010-04-01 15:37:45|12726|10:10|new|utils.pm:
> run_ssh_command(6276)|SSH
> > > command executed on 192.168.129.37, returning (0, "none")
> > > 392 2010-04-01
> > 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on 192.168.129.37<http://192.168.129.37>:
> > > 393 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22 -x
> > > 192.168.129.37 'cp
> > >
> /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk
> > >
> /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk'
> > > 2>&1
> > > 394 2010-04-01
> > 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > > output:
> > > 395 |12726|10:10|new| none
> > > 396 2010-04-01 15:37:45|12726|10:10|new|utils.pm:
> run_ssh_command(6276)|SSH
> > > command executed on 192.168.129.37, returning (0, "none")
> > > 397 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(278)|COPIED
> VMDK
> > > SUCCESSFULLY
> > > 398 2010-04-01
> > 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on 192.168.129.37<http://192.168.129.37>:
> > > 399 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22 -x
> > > 192.168.129.37 'cp
> > >
> /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk
> > >
> /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk'
> > > 2>&1
> > > 400 2010-04-01 15:37:49|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:37:49
> > > 401 2010-04-01 15:37:54|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:37:54
> > > 402 2010-04-01 15:37:59|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 3/3: executing SSH command on esx35-hypervisor-1:
> > > 403 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > > /etc/ssh/external_sshd_config"' 2>&1
> > > 404 2010-04-01 15:37:59|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:37:59
> > > 405 2010-04-01
> > 15:38:00|12719|9:9|deleted|utils.pm:
> run_ssh_command(6262)|run_ssh_command
> > > output:
> > > 406 |12719|9:9|deleted| ash: pkill: not found
> > > 407 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:
> run_ssh_command(6272)|SSH
> > > command executed on esx35-hypervisor-1, command:
> > > 408 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > > /etc/ssh/external_sshd_config"' 2>&1
> > > 409 |12719|9:9|deleted| returning (127, "ash: pkill: not found")
> > > 410 2010-04-01
> > > 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(916)|output
> detected:
> > > ash: pkill: not found
> > > 411 2010-04-01
> > > 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(921)|ext_sshd on
> > > esx35-hypervisor-1 stopped
> > > 412 2010-04-01
> > > 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(884)|stopped external
> sshd
> > > 413 2010-04-01
> > > 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(888)|access has been
> > > revoked to esx35-hypervisor-1
> > > 414 2010-04-01
> > 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on esx35-hypervisor-1:
> > > 415 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > > 416
> > > 417 |12719|9:9|deleted| ---- WARNING ----
> > > 418 |12719|9:9|deleted| 2010-04-01
> > 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 1/3: failed to execute SSH command on esx35-hypervisor-1:
> /usr/sbin/userdel
> > > admin, exit status: 255, SSH exits with the exit status of the remote
> > > command or with 255 if an error occurred, output:
> > > 419 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/passwd'
> > > 420 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/shadow'
> > > 421 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/group'
> > > 422 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/gshadow'
> > > 423 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 424 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 425 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > > 426 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 427 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 428 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 429 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 430
> > > 431 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 2 seconds before making next SSH attempt
> > > 432 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 2/3: executing SSH command on esx35-hypervisor-1:
> > > 433 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > > 434
> > > 435 |12719|9:9|deleted| ---- WARNING ----
> > > 436 |12719|9:9|deleted| 2010-04-01
> > 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 2/3: failed to execute SSH command on esx35-hypervisor-1:
> /usr/sbin/userdel
> > > admin, exit status: 255, SSH exits with the exit status of the remote
> > > command or with 255 if an error occurred, output:
> > > 437 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/passwd'
> > > 438 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/shadow'
> > > 439 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/group'
> > > 440 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/gshadow'
> > > 441 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 442 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 443 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > > 444 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 445 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 446 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 447 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 448
> > > 449 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 15 seconds before making next SSH attempt
> > > 450 2010-04-01 15:38:04|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:38:04
> > > 451 2010-04-01 15:38:09|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:38:09
> > > 452 2010-04-01 15:38:14|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:38:14
> > > 453 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 3/3: executing SSH command on esx35-hypervisor-1:
> > > 454 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > > 455
> > > 456 |12719|9:9|deleted| ---- WARNING ----
> > > 457 |12719|9:9|deleted| 2010-04-01
> > 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 3/3: failed to execute SSH command on esx35-hypervisor-1:
> /usr/sbin/userdel
> > > admin, exit status: 255, SSH exits with the exit status of the remote
> > > command or with 255 if an error occurred, output:
> > > 458 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/passwd'
> > > 459 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/shadow'
> > > 460 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/group'
> > > 461 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/gshadow'
> > > 462 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 463 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 464 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > > 465 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 466 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 467 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 468 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 469
> > > 470
> > > 471 |12719|9:9|deleted| ---- WARNING ----
> > > 472 |12719|9:9|deleted| 2010-04-01
> > 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1
> '/usr/sbin/userdel
> > > admin' 2>&1, exit status: 255, output:
> > > 473 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/passwd'
> > > 474 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/shadow'
> > > 475 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/group'
> > > 476 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/gshadow'
> > > 477 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 478 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6285)
> > > 479 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > > 480 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 481 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 482 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 483 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 484
> > > 485 2010-04-01
> > 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on esx35-hypervisor-1:
> > > 486 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > > 487
> > > 488 |12719|9:9|deleted| ---- WARNING ----
> > > 489 |12719|9:9|deleted| 2010-04-01
> > 15:38:18|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 1/3: failed to execute SSH command on esx35-hypervisor-1:
> /usr/sbin/userdel
> > > admin, exit status: 255, SSH exits with the exit status of the remote
> > > command or with 255 if an error occurred, output:
> > > 490 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/passwd'
> > > 491 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/shadow'
> > > 492 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/group'
> > > 493 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/gshadow'
> > > 494 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 495 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 496 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > > 497 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 498 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 499 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 500 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 501
> > > 502 2010-04-01 15:38:18|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 2 seconds before making next SSH attempt
> > > 503 2010-04-01 15:38:19|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:38:19
> > > 504 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 2/3: executing SSH command on esx35-hypervisor-1:
> > > 505 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > > 506
> > > 507 |12719|9:9|deleted| ---- WARNING ----
> > > 508 |12719|9:9|deleted| 2010-04-01
> > 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 2/3: failed to execute SSH command on esx35-hypervisor-1:
> /usr/sbin/userdel
> > > admin, exit status: 255, SSH exits with the exit status of the remote
> > > command or with 255 if an error occurred, output:
> > > 509 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/passwd'
> > > 510 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/shadow'
> > > 511 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/group'
> > > 512 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/gshadow'
> > > 513 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 514 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 515 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > > 516 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 517 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 518 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 519 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 520
> > > 521 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 15 seconds before making next SSH attempt
> > > 522 2010-04-01 15:38:25|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:38:24
> > > 523 2010-04-01 15:38:30|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:38:30
> > > 524 2010-04-01 15:38:35|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:38:35
> > > 525 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 3/3: executing SSH command on esx35-hypervisor-1:
> > > 526 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > > 527
> > > 528 |12719|9:9|deleted| ---- WARNING ----
> > > 529 |12719|9:9|deleted| 2010-04-01
> > 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 3/3: failed to execute SSH command on esx35-hypervisor-1:
> /usr/sbin/userdel
> > > admin, exit status: 255, SSH exits with the exit status of the remote
> > > command or with 255 if an error occurred, output:
> > > 530 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/passwd'
> > > 531 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/shadow'
> > > 532 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/group'
> > > 533 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/gshadow'
> > > 534 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 535 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 536 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > > 537 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 538 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 539 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 540 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 541
> > > 542
> > > 543 |12719|9:9|deleted| ---- WARNING ----
> > > 544 |12719|9:9|deleted| 2010-04-01
> > 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1
> '/usr/sbin/userdel
> > > admin' 2>&1, exit status: 255, output:
> > > 545 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/passwd'
> > > 546 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/shadow'
> > > 547 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/group'
> > > 548 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't
> find
> > > 'admin' in '/etc/gshadow'
> > > 549 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 550 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6285)
> > > 551 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > > 552 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 553 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 554 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 555 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 556
> > > 557 2010-04-01
> > 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on esx35-hypervisor-1:
> > > 558 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> > > /etc/ssh/external_sshd_config' 2>&1
> > > 559
> > > 560 |12719|9:9|deleted| ---- WARNING ----
> > > 561 |12719|9:9|deleted| 2010-04-01
> > 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> > > "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255,
> SSH
> > > exits with the exit status of the remote command or with 255 if an
> error
> > > occurred, output:
> > > 562 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > > /etc/ssh/external_sshd_config: No such file or directory
> > > 563 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 564 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 565 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > > 566 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 567 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 568 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 569 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 570
> > > 571 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 2 seconds before making next SSH attempt
> > > 572 2010-04-01 15:38:37|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 2/3: executing SSH command on esx35-hypervisor-1:
> > > 573 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> > > /etc/ssh/external_sshd_config' 2>&1
> > > 574
> > > 575 |12719|9:9|deleted| ---- WARNING ----
> > > 576 |12719|9:9|deleted| 2010-04-01
> > 15:38:38|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> > > "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255,
> SSH
> > > exits with the exit status of the remote command or with 255 if an
> error
> > > occurred, output:
> > > 577 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > > /etc/ssh/external_sshd_config: No such file or directory
> > > 578 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 579 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 580 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > > 581 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 582 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 583 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 584 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 585
> > > 586 2010-04-01 15:38:38|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 15 seconds before making next SSH attempt
> > > 587 2010-04-01 15:38:40|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:38:40
> > > 588 2010-04-01 15:38:45|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:38:45
> > > 589 2010-04-01 15:38:50|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:38:50
> > > 590 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 3/3: executing SSH command on esx35-hypervisor-1:
> > > 591 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> > > /etc/ssh/external_sshd_config' 2>&1
> > > 592
> > > 593 |12719|9:9|deleted| ---- WARNING ----
> > > 594 |12719|9:9|deleted| 2010-04-01
> > 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> > > "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255,
> SSH
> > > exits with the exit status of the remote command or with 255 if an
> error
> > > occurred, output:
> > > 595 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > > /etc/ssh/external_sshd_config: No such file or directory
> > > 596 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 597 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 598 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > > 599 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 600 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 601 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 602 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 603
> > > 604
> > > 605 |12719|9:9|deleted| ---- WARNING ----
> > > 606 |12719|9:9|deleted| 2010-04-01
> > 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie
> "/^AllowUsers
> > > .*/d" /etc/ssh/external_sshd_config' 2>&1, exit status: 255, output:
> > > 607 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > > /etc/ssh/external_sshd_config: No such file or directory
> > > 608 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 609 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6285)
> > > 610 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > > 611 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 612 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 613 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 614 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 615
> > > 616 No recipient addresses found in header
> > > 617 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:
> mail(1348)|SUCCESS
> > > -- Sending mail To: , PROBLEM -- Linux.pm
> > > 618
> > > 619 |12719|9:9|deleted| ---- CRITICAL ----
> > > 620 |12719|9:9|deleted| 2010-04-01
> > > 15:38:53|12719|9:9|deleted|Linux.pm:delete_user(566)|failed to add
> > > AllowUsers admin to external_sshd_config
> > > 621 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 622 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 566)
> > > 623 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
> > > 624 |12719|9:9|deleted| (-3) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 625 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
> > > 626 |12719|9:9|deleted| (-5) vcld, main (line: 341)
> > > 627
> > > 628 2010-04-01
> > 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on esx35-hypervisor-1:
> > > 629 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> > > 630
> > > 631 |12719|9:9|deleted| ---- WARNING ----
> > > 632 |12719|9:9|deleted| 2010-04-01
> > 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> "/^admin
> > > .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of
> the
> > > remote command or with 255 if an error occurred, output:
> > > 633 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> /etc/sudoers:
> > > No such file or directory
> > > 634 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 635 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 636 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > > 637 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 638 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 639 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 640 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 641
> > > 642 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 2 seconds before making next SSH attempt
> > > 643 2010-04-01 15:38:55|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:38:55
> > > 644 2010-04-01 15:38:55|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 2/3: executing SSH command on esx35-hypervisor-1:
> > > 645 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> > > 646
> > > 647 |12719|9:9|deleted| ---- WARNING ----
> > > 648 |12719|9:9|deleted| 2010-04-01
> > 15:38:56|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> "/^admin
> > > .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of
> the
> > > remote command or with 255 if an error occurred, output:
> > > 649 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> /etc/sudoers:
> > > No such file or directory
> > > 650 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 651 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 652 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > > 653 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 654 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 655 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 656 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 657
> > > 658 2010-04-01 15:38:56|12719|9:9|deleted|utils.pm:
> run_ssh_command(6172)|sleeping
> > > for 15 seconds before making next SSH attempt
> > > 659 2010-04-01 15:39:00|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:00
> > > 660 2010-04-01 15:39:05|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:05
> > > 661 2010-04-01 15:39:10|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:10
> > > 662 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:
> run_ssh_command(6184)|attempt
> > > 3/3: executing SSH command on esx35-hypervisor-1:
> > > 663 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22
> > > -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> > > 664
> > > 665 |12719|9:9|deleted| ---- WARNING ----
> > > 666 |12719|9:9|deleted| 2010-04-01
> > 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > > 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> "/^admin
> > > .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of
> the
> > > remote command or with 255 if an error occurred, output:
> > > 667 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> /etc/sudoers:
> > > No such file or directory
> > > 668 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 669 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6252)
> > > 670 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > > 671 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 672 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 673 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 674 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 675
> > > 676
> > > 677 |12719|9:9|deleted| ---- WARNING ----
> > > 678 |12719|9:9|deleted| 2010-04-01
> > 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^admin
> > > .*/d" /etc/sudoers' 2>&1, exit status: 255, output:
> > > 679 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> /etc/sudoers:
> > > No such file or directory
> > > 680 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 681 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>,
> run_ssh_command (line: 6285)
> > > 682 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > > 683 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > > 684 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 685 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > > 686 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > > 687
> > > 688 No recipient addresses found in header
> > > 689 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:
> mail(1348)|SUCCESS
> > > -- Sending mail To: , PROBLEM -- Linux.pm
> > > 690
> > > 691 |12719|9:9|deleted| ---- CRITICAL ----
> > > 692 |12719|9:9|deleted| 2010-04-01
> > > 15:39:11|12719|9:9|deleted|Linux.pm:delete_user(578)|failed to clear
> admin
> > > from /etc/sudoers
> > > 693 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify
> (line: 737)
> > > 694 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 578)
> > > 695 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
> > > 696 |12719|9:9|deleted| (-3) reclaim.pm<http://reclaim.pm>, process
> (line: 173)
> > > 697 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
> > > 698 |12719|9:9|deleted| (-5) vcld, main (line: 341)
> > > 699
> > > 700 2010-04-01
> 15:39:11|12719|9:9|deleted|Linux.pm:sanitize(852)|users
> > > have been deleted from esx35-hypervisor-1
> > > 701 2010-04-01 15:39:11|12719|9:9|deleted|reclaim.pm:
> process(173)|esx35-
> > hypervisor-1
> > > has been sanitized
> > > 702 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:
> switch_state(8018)|called
> > > from VCL::reclaim::process(271)
> > > 703 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:
> switch_state(8054)|parent:
> > > parent reservation ID for this request: 9
> > > 704 2010-04-01
> > 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > > entry not inserted, reservation has been deleted
> > > 705 2010-04-01
> > 15:39:11|12719|9:9|deleted|utils.pm:update_request_state(2186)|request
> > > 9 state updated to: complete, laststate to: deleted
> > > 706 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:
> switch_state(8078)|req=9:
> > > request state changed: deleted->complete, laststate: reserved->deleted
> > > 707 2010-04-01
> > 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > > entry not inserted, reservation has been deleted
> > > 708 2010-04-01
> > 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > > entry not inserted, reservation has been deleted
> > > 709 2010-04-01
> > 15:39:11|12719|9:9|deleted|utils.pm:update_computer_state(2228)|computer
> > > 2 state updated to: available
> > > 710 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:
> switch_state(8096)|req=9:
> > > computer esx35-hypervisor-1 state changed: available->available
> > > 711 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:
> switch_state(8106)|req=9:
> > > log table id=10 will not be updated
> > > 712 2010-04-01
> > 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > > entry not inserted, reservation has been deleted
> > > 713 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:
> switch_state(8122)|req=9:
> > > process exiting
> > > 714 2010-04-01
> > > 15:39:11|12719|9:9|deleted|State.pm:DESTROY(905)|destructor called,
> > > ref($self)=VCL::reclaim
> > > 715 2010-04-01
> > 15:39:11|12719|9:9|deleted|utils.pm:
> delete_computerloadlog_reservation(7551)|removing
> > > computerloadlog entries matching loadstate = begin
> > > 716 2010-04-01
> > 15:39:11|12719|9:9|deleted|utils.pm:
> delete_computerloadlog_reservation(7598)|deleted
> > > rows from computerloadlog for reservation id=9
> > > 717 2010-04-01
> 15:39:11|12719|9:9|deleted|State.pm:DESTROY(912)|removed
> > > computerloadlog rows with loadstate=begin for reservation
> > > 718 2010-04-01
> 15:39:11|12719|9:9|deleted|State.pm:DESTROY(924)|number
> > > of database handles state process created: 1
> > > 719 2010-04-01
> 15:39:11|12719|9:9|deleted|State.pm:DESTROY(933)|process
> > > has a database handle stored in $ENV{dbh}, attempting disconnect
> > > 720 2010-04-01
> > > 15:39:11|12719|9:9|deleted|State.pm:DESTROY(935)|$ENV{dbh}: database
> > > disconnect successful
> > > 721 2010-04-01
> > > 15:39:11|12719|9:9|deleted|State.pm:DESTROY(949)|VCL::reclaim process
> 12719
> > > exiting
> > > 722 2010-04-01 15:39:11|5754|vcld:REAPER(744)|VCL process exited for
> > > reservation 9
> > > 723 2010-04-01 15:39:11|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:11
> > > 724 2010-04-01 15:39:11|5754|9:9|complete|vcld:main(250)|request
> deleted
> > > 725 2010-04-01 15:39:16|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:16
> > > 726 2010-04-01 15:39:21|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:21
> > > 727 2010-04-01 15:39:26|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:26
> > > 728 2010-04-01 15:39:31|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:31
> > > 729 2010-04-01 15:39:36|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:36
> > > 730 2010-04-01 15:39:42|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:42
> > > 731 2010-04-01 15:39:47|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:47
> > > 732 2010-04-01 15:39:52|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:52
> > > 733 2010-04-01
> > 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > > output:
> > > 734 |12726|10:10|new| none
> > > 735 2010-04-01 15:39:55|12726|10:10|new|utils.pm:
> run_ssh_command(6276)|SSH
> > > command executed on 192.168.129.37, returning (0, "none")
> > > 736 2010-04-01
> > 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > > SSH command on 192.168.129.37<http://192.168.129.37>:
> > > 737 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p
> 22 -x
> > > 192.168.129.37 'grep adapterType
> > >
> /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk
> > > 2>&1' 2>&1
> > > 738 2010-04-01
> > 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > > output:
> > > 739 |12726|10:10|new| ddb.adapterType = "buslogic"
> > > 740 2010-04-01 15:39:55|12726|10:10|new|utils.pm:
> run_ssh_command(6276)|SSH
> > > command executed on 192.168.129.37, returning (0, "ddb.adapterType =
> > > "buslogic"")
> > > 741 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(311)|adapter=
> > > buslogic
> > > 742 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(342)|eth0 MAC
> > > address set for vcl assigned
> > > 743 Use of uninitialized value in concatenation (.) or string at
> > > 744
> /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pmline 343 (#1)
> > > 745 (W uninitialized) An undefined value was used as if it were
> > > already
> > > 746 defined. It was interpreted as a "" or a 0, but maybe it
> was a
> > > mistake.
> > > 747 To suppress this warning assign a defined value to your
> > > variables.
> > > 748
> > > 749 To help you figure out what was undefined, perl tells you
> what
> > > operation
> > > 750 you used the undefined value in. Note, however, that perl
> > > optimizes your
> > > 751 program and the operation displayed in the warning may not
> > > necessarily
> > > 752 appear literally in your program. For example, "that $foo"
> is
> > > 753 usually optimized into "that " . $foo, and the warning will
> > > refer to
> > > 754 the concatenation (.) operator, even though there is no . in
> > > your
> > > 755 program.
> > > 756
> > > 757
> > > 758 |12726|10:10|new| ---- WARNING ----
> > > 759 |12726|10:10|new| 2010-04-01
> > > 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> > > value in concatenation (.) or string at
> > > /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm<http://esx.pm>
> line 343.
> > > 760 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 761 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> > > 762 |12726|10:10|new| (-2) esx.pm<http://esx.pm>, load (line: 343)
> > > 763 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image
> (line: 665)
> > > 764 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line:
> 266)
> > > 765 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > > 766 |12726|10:10|new| (-6) vcld, main (line: 341)
> > > 767
> > > 768
> > > 769 |12726|10:10|new| ---- WARNING ----
> > > 770 |12726|10:10|new| 2010-04-01
> > > 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> > > value in concatenation (.) or string at
> > > /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm<http://esx.pm>
> line 353.
> > > 771 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 772 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> > > 773 |12726|10:10|new| (-2) esx.pm<http://esx.pm>, load (line: 353)
> > > 774 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image
> (line: 665)
> > > 775 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line:
> 266)
> > > 776 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > > 777 |12726|10:10|new| (-6) vcld, main (line: 341)
> > > 778
> > > 779 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(353)|eth1 MAC
> > > address set for vcl assigned
> > > 780
> > > 781 |12726|10:10|new| ---- WARNING ----
> > > 782 |12726|10:10|new| 2010-04-01
> > > 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> > > value in concatenation (.) or string at
> > > /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm<http://esx.pm>
> line 354.
> > > 783 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 784 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> > > 785 |12726|10:10|new| (-2) esx.pm<http://esx.pm>, load (line: 354)
> > > 786 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image
> (line: 665)
> > > 787 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line:
> 266)
> > > 788 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > > 789 |12726|10:10|new| (-6) vcld, main (line: 341)
> > > 790
> > > 791 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(376)|wrote
> vmxarray
> > > to /tmp/esx3-basewinxp-v0.vmx
> > > 792 2010-04-01 15:39:55|12726|10:10|new|utils.pm:
> run_scp_command(6403)|attempt
> > > 1/3: executing SCP command: /usr/bin/scp -B -i /etc/vcl/vcl.key -P 22
> -p -r
> > > /tmp/esx3-basewinxp-v0.vmx 192.168.129.37:
> /install/image/vmware_images/inuse/esx3-
> > basewinxp-v0/esx3-basewinxp-v0.vmx
> > > 2>&1
> > > 793 2010-04-01 15:39:55|12726|10:10|new|utils.pm:
> run_scp_command(6453)|scp
> > > successful: attempt 1/3, exit status: 0, output: none
> > > 794 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(401)|Register
> > > Command: /usr/lib/vmware-vcli/apps/vm/vmregister.pl<
> http://vmregister.pl> --server
> > > 'esx35-hypervisor-1' --username root --password 'xxxxxxxxxxx' --vmxpath
> > > '[VCL]/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmx' --operation
> register
> > > --vmname esx3-basewinxp-v0 --pool Resources --hostname '
> > > esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu>'
> --datacenter 'ha-datacenter'
> > > 795 2010-04-01 15:39:57|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:39:57
> > > 796 2010-04-01 15:39:57|12726|10:10|new|esx.pm:
> load(404)|Registered:
> > > 797 |12726|10:10|new| SOAP Fault:
> > > 798 |12726|10:10|new| -----------
> > > 799 |12726|10:10|new| Fault string: fault.RestrictedVersion.summary
> > > 800 |12726|10:10|new| Fault detail: RestrictedVersionFault
> > > 801 2010-04-01 15:39:57|12726|10:10|new|esx.pm:load(413)|Power on
> > > command: /usr/lib/vmware-vcli/apps/vm/vmcontrol.pl<http://vmcontrol.pl>
> --server
> > > 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --operation poweron
> > > --username root --password 'xxxxxxxxxxx'
> > > 802 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(416)|Powered
> on:
> > > Virtual Machine esx3-basewinxp-v0 not found.
> > > 803 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(505)|IP is
> known
> > > for esx3-basewinxp-v0
> > > 804 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(511)|Waiting
> for
> > > ssh to come up on esx3-basewinxp-v0
> > > 805 2010-04-01 15:40:00|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 1
> > > 806 2010-04-01 15:40:02|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:02
> > > 807 2010-04-01 15:40:06|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 2
> > > 808 2010-04-01 15:40:07|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:07
> > > 809 2010-04-01 15:40:12|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:12
> > > 810 2010-04-01 15:40:12|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 3
> > > 811 2010-04-01 15:40:17|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:17
> > > 812 2010-04-01 15:40:18|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 4
> > > 813 2010-04-01 15:40:22|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:22
> > > 814 2010-04-01 15:40:24|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 5
> > > 815 2010-04-01 15:40:27|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:27
> > > 816 2010-04-01 15:40:30|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 6
> > > 817 2010-04-01 15:40:32|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:32
> > > 818 2010-04-01 15:40:36|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 7
> > > 819 2010-04-01 15:40:37|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:37
> > > 820 2010-04-01 15:40:42|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:42
> > > 821 2010-04-01 15:40:42|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 8
> > > 822 2010-04-01 15:40:47|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:47
> > > 823 2010-04-01 15:40:48|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 9
> > > 824 2010-04-01 15:40:52|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:52
> > > 825 2010-04-01 15:40:54|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 10
> > > 826 2010-04-01 15:40:57|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:40:57
> > > 827 2010-04-01 15:41:00|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 11
> > > 828 2010-04-01 15:41:02|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:02
> > > 829 2010-04-01 15:41:06|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 12
> > > 830 2010-04-01 15:41:07|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:07
> > > 831 2010-04-01 15:41:12|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:12
> > > 832 2010-04-01 15:41:12|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 13
> > > 833 2010-04-01 15:41:17|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:17
> > > 834 2010-04-01 15:41:18|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 14
> > > 835 2010-04-01 15:41:22|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:22
> > > 836 2010-04-01 15:41:24|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 15
> > > 837 2010-04-01 15:41:27|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:27
> > > 838 2010-04-01 15:41:30|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 16
> > > 839 2010-04-01 15:41:32|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:32
> > > 840 2010-04-01 15:41:36|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 17
> > > 841 2010-04-01 15:41:37|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:37
> > > 842 2010-04-01 15:41:42|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:42
> > > 843 2010-04-01 15:41:42|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 18
> > > 844 2010-04-01 15:41:47|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:47
> > > 845 2010-04-01 15:41:48|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 19
> > > 846 2010-04-01 15:41:52|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:52
> > > 847 2010-04-01 15:41:54|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 20
> > > 848 2010-04-01 15:41:57|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:41:57
> > > 849 2010-04-01 15:42:01|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 21
> > > 850 2010-04-01 15:42:02|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:02
> > > 851 2010-04-01 15:42:07|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 22
> > > 852 2010-04-01 15:42:07|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:07
> > > 853 2010-04-01 15:42:12|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:12
> > > 854 2010-04-01 15:42:13|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 23
> > > 855 2010-04-01 15:42:18|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:18
> > > 856 2010-04-01 15:42:19|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 24
> > > 857 2010-04-01 15:42:23|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:23
> > > 858 2010-04-01 15:42:25|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 25
> > > 859 2010-04-01 15:42:28|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:28
> > > 860 2010-04-01 15:42:31|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 26
> > > 861 2010-04-01 15:42:33|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:33
> > > 862 2010-04-01 15:42:37|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 27
> > > 863 2010-04-01 15:42:38|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:38
> > > 864 2010-04-01 15:42:43|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 28
> > > 865 2010-04-01 15:42:43|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:43
> > > 866 2010-04-01 15:42:48|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:48
> > > 867 2010-04-01 15:42:49|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 29
> > > 868 2010-04-01 15:42:53|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:53
> > > 869 2010-04-01 15:42:55|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 30
> > > 870 2010-04-01 15:42:58|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:42:58
> > > 871 2010-04-01 15:43:01|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 31
> > > 872 2010-04-01 15:43:03|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:03
> > > 873 2010-04-01 15:43:07|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 32
> > > 874 2010-04-01 15:43:08|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:08
> > > 875 2010-04-01 15:43:13|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 33
> > > 876 2010-04-01 15:43:13|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:13
> > > 877 2010-04-01 15:43:18|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:18
> > > 878 2010-04-01 15:43:19|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 34
> > > 879 2010-04-01 15:43:23|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:23
> > > 880 2010-04-01 15:43:25|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 35
> > > 881 2010-04-01 15:43:28|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:28
> > > 882 2010-04-01 15:43:31|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 36
> > > 883 2010-04-01 15:43:33|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:33
> > > 884 2010-04-01 15:43:37|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 37
> > > 885 2010-04-01 15:43:38|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:38
> > > 886 2010-04-01 15:43:43|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 38
> > > 887 2010-04-01 15:43:43|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:43
> > > 888 2010-04-01 15:43:48|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:48
> > > 889 2010-04-01 15:43:49|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 39
> > > 890 2010-04-01 15:43:53|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:53
> > > 891 2010-04-01 15:43:55|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 40
> > > 892 2010-04-01 15:43:58|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:43:58
> > > 893 2010-04-01 15:44:01|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 41
> > > 894 2010-04-01 15:44:03|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:03
> > > 895 2010-04-01 15:44:07|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 42
> > > 896 2010-04-01 15:44:08|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:08
> > > 897 2010-04-01 15:44:13|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 43
> > > 898 2010-04-01 15:44:13|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:13
> > > 899 2010-04-01 15:44:18|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:18
> > > 900 2010-04-01 15:44:19|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 44
> > > 901 2010-04-01 15:44:23|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:23
> > > 902 2010-04-01 15:44:25|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 45
> > > 903 2010-04-01 15:44:29|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:28
> > > 904 2010-04-01 15:44:31|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 46
> > > 905 2010-04-01 15:44:34|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:34
> > > 906 2010-04-01 15:44:37|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 47
> > > 907 2010-04-01 15:44:39|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:39
> > > 908 2010-04-01 15:44:43|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 48
> > > 909 2010-04-01 15:44:44|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:44
> > > 910 2010-04-01 15:44:49|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:49
> > > 911 2010-04-01 15:44:49|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 49
> > > 912 2010-04-01 15:44:54|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:54
> > > 913 2010-04-01 15:44:55|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 50
> > > 914 2010-04-01 15:44:59|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:44:59
> > > 915 2010-04-01 15:45:01|12726|10:10|new|esx.pm:load(528)|going to
> sleep
> > > 5 seconds, waiting for computer to start SSH. Try 51
> > > 916 2010-04-01 15:45:04|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:45:04
> > > 917 No recipient addresses found in header
> > > 918 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS
> --
> > > Sending mail To: , PROBLEM -- esx.pm<http://esx.pm>
> > > 919
> > > 920 |12726|10:10|new| ---- CRITICAL ----
> > > 921 |12726|10:10|new| 2010-04-01 15:45:07|12726|10:10|new|esx.pm:
> load(521)|waited
> > > acceptable amount of time for sshd to become active, please check
> > > esx3-basewinxp-v0 on esx35-hypervisor-1
> > > 922 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 923 |12726|10:10|new| (-1) esx.pm<http://esx.pm>, load (line: 521)
> > > 924 |12726|10:10|new| (-2) new.pm<http://new.pm>, reload_image
> (line: 665)
> > > 925 |12726|10:10|new| (-3) new.pm<http://new.pm>, process (line:
> 266)
> > > 926 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> > > 927 |12726|10:10|new| (-5) vcld, main (line: 341)
> > > 928
> > > 929
> > > 930 |12726|10:10|new| ---- WARNING ----
> > > 931 |12726|10:10|new| 2010-04-01
> > 15:45:08|12726|10:10|new|new.pm:reload_image(670)|esx3-basewinxp-v0
> > > failed to load on esx3-basewinxp-v0, returning
> > > 932 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 933 |12726|10:10|new| (-1) new.pm<http://new.pm>, reload_image
> (line: 670)
> > > 934 |12726|10:10|new| (-2) new.pm<http://new.pm>, process (line:
> 266)
> > > 935 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
> > > 936 |12726|10:10|new| (-4) vcld, main (line: 341)
> > > 937
> > > 938 2010-04-01 15:45:08|12726|10:10|new|utils.pm:
> insertloadlog(4710)|inserted
> > > computer=3, loadimagefailed, esx3-basewinxp-v0 failed to load on
> > > esx3-basewinxp-v0
> > > 939
> > > 940 |12726|10:10|new| ---- WARNING ----
> > > 941 |12726|10:10|new| 2010-04-01
> > 15:45:08|12726|10:10|new|new.pm:process(313)|failed
> > > to load esx3-basewinxp-v0 with esx3-basewinxp-v0
> > > 942 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 943 |12726|10:10|new| (-1) new.pm<http://new.pm>, process (line:
> 313)
> > > 944 |12726|10:10|new| (-2) vcld, make_new_child (line: 594)
> > > 945 |12726|10:10|new| (-3) vcld, main (line: 341)
> > > 946
> > > 947 2010-04-01
> > >
> 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > > to retrieve current state of computer esx3-basewinxp-v0 from the
> database
> > > 948 2010-04-01
> > >
> 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > > current state of computer esx3-basewinxp-v0 from the database:
> reloading
> > > 949 2010-04-01
> > > 15:45:08|12726|10:10|new|DataStructure.pm:_automethod(697)|data
> structure
> > > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > > 950 |12726|10:10|new| computer_state_name = reloading
> > > 951 No recipient addresses found in header
> > > 952 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS
> --
> > > Sending mail To: , PROBLEM -- State.pm
> > > 953
> > > 954 |12726|10:10|new| ---- CRITICAL ----
> > > 955 |12726|10:10|new| 2010-04-01
> > > 15:45:08|12726|10:10|new|State.pm:reservation_failed(290)|reservation
> failed
> > > on esx3-basewinxp-v0: process failed after trying to load or make
> available
> > > 956 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line:
> 737)
> > > 957 |12726|10:10|new| (-1) State.pm, reservation_failed (line: 290)
> > > 958 |12726|10:10|new| (-2) new.pm<http://new.pm>, process (line:
> 316)
> > > 959 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
> > > 960 |12726|10:10|new| (-4) vcld, main (line: 341)
> > > 961
> > > 962 2010-04-01 15:45:08|12726|10:10|new|utils.pm:
> insertloadlog(4710)|inserted
> > > computer=3, failed, process failed after trying to load or make
> available
> > > 963 2010-04-01
> > > 15:45:08|12726|10:10|new|State.pm:reservation_failed(293)|inserted
> > > computerloadlog entry
> > > 964 2010-04-01
> > > 15:45:08|12726|10:10|new|State.pm:reservation_failed(301)|updated log
> ending
> > > value to 'failed', logid=11
> > > 965 2010-04-01
> > 15:45:08|12726|10:10|new|utils.pm:update_computer_state(2228)|computer
> > > 3 state updated to: failed
> > > 966 2010-04-01
> > > 15:45:08|12726|10:10|new|State.pm:reservation_failed(312)|computer
> > > esx3-basewinxp-v0 (3) state set to failed
> > > 967 2010-04-01 15:45:08|12726|10:10|new|utils.pm:
> update_request_state(2186)|request
> > > 10 state updated to: failed, laststate to: new
> > > 968 2010-04-01
> > > 15:45:08|12726|10:10|new|State.pm:reservation_failed(325)|set request
> state
> > > to 'failed'/'new'
> > > 969 2010-04-01 15:45:08|12726|10:10|new|utils.pm:
> is_inblockrequest(6972)|zero
> > > rows were returned from database select
> > > 970 2010-04-01
> > >
> 15:45:08|12726|10:10|new|State.pm:reservation_failed(343)|esx3-basewinxp-v0
> > > is NOT in blockcomputers table
> > > 971 2010-04-01
> > > 15:45:08|12726|10:10|new|State.pm:reservation_failed(346)|exiting 1
> > > 972 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:DESTROY(905)|destructor
> > > called, ref($self)=VCL::new
> > > 973 2010-04-01
> > 15:45:08|12726|10:10|new|utils.pm:
> delete_computerloadlog_reservation(7551)|removing
> > > computerloadlog entries matching loadstate = begin
> > > 974 2010-04-01
> > 15:45:08|12726|10:10|new|utils.pm:
> delete_computerloadlog_reservation(7598)|deleted
> > > rows from computerloadlog for reservation id=10
> > > 975 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:DESTROY(912)|removed
> > > computerloadlog rows with loadstate=begin for reservation
> > > 976 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(924)|number
> of
> > > database handles state process created: 1
> > > 977 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:DESTROY(933)|process
> > > has a database handle stored in $ENV{dbh}, attempting disconnect
> > > 978 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:DESTROY(935)|$ENV{dbh}:
> > > database disconnect successful
> > > 979 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:DESTROY(949)|VCL::new
> > > process 12726 exiting
> > > 980 2010-04-01 15:45:08|5754|vcld:REAPER(744)|VCL process exited for
> > > reservation 10
> > > 981 2010-04-01 15:45:08|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:45:08
> > > 982 2010-04-01 15:45:13|5754|vcld:main(165)|lastcheckin time updated
> for
> > > management node 1: 2010-04-01 15:45:13
> > >
> > >
> > ================================================================
> > =====================
> > >
> > > Thank you,
> > > Amit
> > >
> > >
> > >
>
>
>
RE: VCL Getting Reservation to work
Posted by "Kumar, Amit H." <AH...@odu.edu>.
Hi Ryan,
Yes my ESXi server was ahead of all other computers that were available, Once I changed the state of the ESXi server to inuse. The next available VMguest computer was picked up and the process continued and worked.
Also we noticed that the license we had put in for the ESXi disabled remote controlling of the VMguest using the perl toolkit API/vshpere;
Interestingly when we changed back to the evaluation version the VMgues was registering on the VMhost correctly and was now able to power on as well.
Now at the last mile Stone: Though I can directly ssh, to my vmguest that is powered on, from management node: The reservation process fails because it complains that it waiting enough time to ssh to the Vmguest and couldn't. I noticed that when I tried to SSH manually from the management host to the vmgues powered on during the reservation process, there was a significant delay in ssh response, I would say about 20-30secs. Not sure if this could cause the problem or not. I looked at the run_ssh_command code, I see it progressively increases the delay so not sure what is missing here ?????
Hope any of you can shed some more help on me ...
Last few lined of the log:
<snip>
No recipient addresses found in header
2010-04-13 12:39:38|13024|23:23|new|utils.pm:mail(1348)|SUCCESS -- Sending mail To: , PROBLEM -- esx.pm
|13024|23:23|new| ---- CRITICAL ----
|13024|23:23|new| 2010-04-13 12:39:37|13024|23:23|new|esx.pm:load(521)|waited acceptable amount of time for sshd to become active, please check esx3-basewinxp-v0 on esx35-hypervisor-1
|13024|23:23|new| ( 0) utils.pm, notify (line: 737)
|13024|23:23|new| (-1) esx.pm, load (line: 521)
|13024|23:23|new| (-2) new.pm, reload_image (line: 665)
|13024|23:23|new| (-3) new.pm, process (line: 266)
|13024|23:23|new| (-4) vcld, make_new_child (line: 594)
|13024|23:23|new| (-5) vcld, main (line: 341)
|13024|23:23|new| ---- WARNING ----
|13024|23:23|new| 2010-04-13 12:39:38|13024|23:23|new|new.pm:reload_image(670)|esx3-basewinxp-v0 failed to load on esx3-basewinxp-v0, returning
|13024|23:23|new| ( 0) utils.pm, notify (line: 737)
|13024|23:23|new| (-1) new.pm, reload_image (line: 670)
|13024|23:23|new| (-2) new.pm, process (line: 266)
|13024|23:23|new| (-3) vcld, make_new_child (line: 594)
|13024|23:23|new| (-4) vcld, main (line: 341)
2010-04-13 12:39:38|13024|23:23|new|utils.pm:insertloadlog(4710)|inserted computer=3, loadimagefailed, esx3-basewinxp-v0 failed to load on esx3-basewinxp-v0
|13024|23:23|new| ---- WARNING ----
|13024|23:23|new| 2010-04-13 12:39:38|13024|23:23|new|new.pm:process(313)|failed to load esx3-basewinxp-v0 with esx3-basewinxp-v0
|13024|23:23|new| ( 0) utils.pm, notify (line: 737)
|13024|23:23|new| (-1) new.pm, process (line: 313)
|13024|23:23|new| (-2) vcld, make_new_child (line: 594)
|13024|23:23|new| (-3) vcld, main (line: 341)
2010-04-13 12:39:38|13024|23:23|new|DataStructure.pm:get_computer_state_name(1946)|attempting to retrieve current state of computer esx3-basewinxp-v0 from the database
2010-04-13 12:39:38|13024|23:23|new|DataStructure.pm:get_computer_state_name(1977)|retrieved current state of computer esx3-basewinxp-v0 from the database: reloading
2010-04-13 12:39:38|13024|23:23|new|DataStructure.pm:_automethod(697)|data structure updated: $self->request_data->{reservation}{23}{computer}{state}{name}
|13024|23:23|new| computer_state_name = reloading
No recipient addresses found in header
2010-04-13 12:39:38|13024|23:23|new|utils.pm:mail(1348)|SUCCESS -- Sending mail To: , PROBLEM -- State.pm
|13024|23:23|new| ---- CRITICAL ----
|13024|23:23|new| 2010-04-13 12:39:38|13024|23:23|new|State.pm:reservation_failed(290)|reservation failed on esx3-basewinxp-v0: process failed after trying to load or make available
|13024|23:23|new| ( 0) utils.pm, notify (line: 737)
|13024|23:23|new| (-1) State.pm, reservation_failed (line: 290)
|13024|23:23|new| (-2) new.pm, process (line: 316)
|13024|23:23|new| (-3) vcld, make_new_child (line: 594)
|13024|23:23|new| (-4) vcld, main (line: 341)
2010-04-13 12:39:38|13024|23:23|new|utils.pm:insertloadlog(4710)|inserted computer=3, failed, process failed after trying to load or make available
2010-04-13 12:39:38|13024|23:23|new|State.pm:reservation_failed(293)|inserted computerloadlog entry
2010-04-13 12:39:38|13024|23:23|new|State.pm:reservation_failed(301)|updated log ending value to 'failed', logid=22
2010-04-13 12:39:38|13024|23:23|new|utils.pm:update_computer_state(2228)|computer 3 state updated to: failed
2010-04-13 12:39:38|13024|23:23|new|State.pm:reservation_failed(312)|computer esx3-basewinxp-v0 (3) state set to failed
2010-04-13 12:39:38|13024|23:23|new|utils.pm:update_request_state(2186)|request 23 state updated to: failed, laststate to: new
2010-04-13 12:39:38|13024|23:23|new|State.pm:reservation_failed(325)|set request state to 'failed'/'new'
2010-04-13 12:39:38|13024|23:23|new|utils.pm:is_inblockrequest(6972)|zero rows were returned from database select
2010-04-13 12:39:38|13024|23:23|new|State.pm:reservation_failed(343)|esx3-basewinxp-v0 is NOT in blockcomputers table
2010-04-13 12:39:38|13024|23:23|new|State.pm:reservation_failed(346)|exiting 1
2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(905)|destructor called, ref($self)=VCL::new
2010-04-13 12:39:38|13024|23:23|new|utils.pm:delete_computerloadlog_reservation(7551)|removing computerloadlog entries matching loadstate = begin
2010-04-13 12:39:38|13024|23:23|new|utils.pm:delete_computerloadlog_reservation(7598)|deleted rows from computerloadlog for reservation id=23
2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(912)|removed computerloadlog rows with loadstate=begin for reservation
2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(924)|number of database handles state process created: 1
2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(933)|process has a database handle stored in $ENV{dbh}, attempting disconnect
2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(935)|$ENV{dbh}: database disconnect successful
2010-04-13 12:39:38|13024|23:23|new|State.pm:DESTROY(949)|VCL::new process 13024 exiting
</snip>
Thank you,
Amit
From: Ryan Johnson [mailto:rjohn06@gwmail.gwu.edu]
Sent: Monday, April 05, 2010 10:27 AM
To: Kumar, Amit H.
Subject: Re: VCL Getting Reservation to work
So for every VM you want to use you have to have a row for it in your computer table. If you are doing manual reservations then you should set the computerid column of the manual reservation you are creating to the id of a VM in the computer table. I think you were passing the id of the ESXi server as the id in the reservation for your test.
On Mon, Apr 5, 2010 at 10:13 AM, Kumar, Amit H. <AH...@odu.edu>> wrote:
Hi Ryan,
I don't have the reservation table for that trial.
Yes I am doing manual reservation. Not sure what do you mean by "specify a vm as the computer id in your reservation row" Could you please elaborate on that.
I do have my vm's assigned to my Vmhosts in the Vmprofile tab. Not sure what additional things we have to do here?
Thank you,
Amit
From: Ryan Johnson [mailto:rjohn06@gwmail.gwu.edu<ma...@gwmail.gwu.edu>]
Sent: Friday, April 02, 2010 3:01 PM
To: Kumar, Amit H.
Subject: Re: [Spam: 6.0] Re: [Spam: 6.0] Re: VCL Getting Reservation to work
I think the problem might have something to do with the computer you specified for the reservation. After looking at the logs, I see that it loads an OS module and then tries to run an ssh command on your hypervisor. If you are using manual reservations, then you should specify a vm as the computer id in your reservation row. If you still have the reservation table info from this could you attach it?. Eventually, you will have to map the vm's in the computer table to the hypervisor you are using.
On Thu, Apr 1, 2010 at 4:48 PM, Kumar, Amit H. <AH...@odu.edu>> wrote:
Attached are the screen shots.
Thank you,
Amit
> -----Original Message-----
> From: Ryan Johnson [mailto:rjohn06@gwmail.gwu.edu<ma...@gwmail.gwu.edu>]
> Sent: Thursday, April 01, 2010 4:23 PM
> To: vcl-dev@incubator.apache.org<ma...@incubator.apache.org>
> Subject: [Spam: 6.0] Re: [Spam: 6.0] Re: VCL Getting Reservation to work
>
> Yea, that is a little hard on the eyes, any chance you could send screen
> shots in a view where the columns are lined up with the column names?
>
> On Thu, Apr 1, 2010 at 4:00 PM, Kumar, Amit H. <AH...@odu.edu>> wrote:
>
> > The Savior! Thank you Ryan getting back!!!
> >
> > Not sure if this is the best format for readability...please let me know
> > otherwise.
> >
> > I am also a recent log that I tried with verbose ssh enabled etc, I haven't
> > looked at it yet. It went past my previous log. .....................
> >
> > Image: Table: Rows: 10
> > ====================
> > id name prettyname ownerid platformid OSid imagemetaid minram
> > minprocnumber minprocspeed minnetwork maxconcurrent reloadtime deleted
> > test lastupdate forcheckout maxinitialtime project size architecture
> > description usage basedoffrevisionid
> > 1 winxp-base1-v0 No Apps (WinXP) 1 [->admin] 1 [->i386] 7 [->winxp] NULL 0
> > 1 0 10 NULL 14 0 0 2007-04-11 16:07:38 1 0 vcl 1045 x86 NULL NULL 0
> > 2 rhel4-base2-v0 Red Hat Enterprise Linux 4.4 Base (KS) 1 [->admin] 1
> > [->i386] 12 [->rhel4] NULL 1024 1 1024 100 NULL 13 0 0 2007-03-02 16:33:33 1
> > 0 vcl 0 x86 NULL NULL 0
> > 3 rh4image-base3-v0 RHEL4 base (image) 1 [->admin] 1 [->i386] 13
> > [->rh4image] NULL 10 1 1024 100 NULL 10 0 0 2007-01-24 15:02:07 1 0 vcl 1450
> > x86 NULL NULL 0
> > 4 noimage No Image 1 [->admin] 1 [->i386] 2 [->sun4x_58] NULL 0 1 0 10 NULL
> > 0 0 0 NULL 1 0 vcl 1450 x86 NULL NULL 0
> > 5 rhfc5-fc5base5-v0 Red Hat Fedora Core 5 base (KS) 1 [->admin] 1 [->i386]
> > 15 [->rhfc5] NULL 1024 1 1024 100 NULL 13 0 0 2006-10-02 10:04:24 1 0 vcl
> > 1450 x86 NULL NULL 0
> > 6 fc5image-FC5baseRHFC56-v0 FC5base(RHFC5) 1 [->admin] 1 [->i386] 14
> > [->fc5image] NULL 64 1 500 10 NULL 10 0 0 2006-10-02 14:13:26 1 0 vcl 0 x86
> > NULL NULL 0
> > 7 vmwarewinxp-base7-v1 No Apps (WinXP vmware) 1 [->admin] 1 [->i386] 16
> > [->vmwarewinxp] NULL 512 1 1024 100 NULL 5 0 0 2007-04-04 09:45:38 1 0 vcl
> > 3244 x86 NULL NULL 0
> > 8 rh4image-VMwareserverhostRHEL48-v3 VMware server host (RHEL4) 1 [->admin]
> > 1 [->i386] 13 [->rh4image] NULL 64 1 500 10 NULL 10 0 0 2007-11-26 11:23:53
> > 1 0 vcl 1450 x86 NULL NULL 0
> > 9 esx35-base-v0 VMware ESX 3.5 standard server 1 [->admin] 1 [->i386] 20
> > [->esx35] NULL 2048 2 2000 100 NULL 9 0 0 2008-03-24 14:23:54 1 0 vcl 1450
> > x86 NULL NULL 0
> > 10 esx3-basewinxp-v0 No Apps (ESXi WinXP) 1 [->admin] 1 [->i386] 20
> > [->esx35] NULL 512 1 1024 100 NULL 9 0 0 2010-03-24 17:35:12 1 0 vcl 4100
> > x86 NULL NULL 0
> > /End Image Table====================
> >
> >
> > ImageRevision Table: Rows: 11
> > ===========================
> > id imageid revision userid datecreated deleted datedeleted
> > production comments imagename
> > 1 1 [->No Apps (WinXP)] 0 1 [->admin] 2007-03-01 14:46:26 0 NULL 1 NULL
> > winxp-base1-v0
> > 2 2 [->Red Hat Enterprise Linux 4.4 Base (KS)] 0 1 [->admin] 2006-09-22
> > 16:33:24 0 NULL 1 NULL rhel4-base2-v0
> > 3 3 [->RHEL4 base (image)] 0 1 [->admin] 2007-01-24 15:04:11 0 NULL 1 NULL
> > rh4image-base3-v0
> > 4 4 [->No Image] 0 1 [->admin] 1980-01-01 00:00:00 0 NULL 1 NULL noimage
> > 5 5 [->Red Hat Fedora Core 5 base (KS)] 0 1 [->admin] 2006-10-02 10:04:24 0
> > NULL 1 NULL rhfc5-fc5base5-v0
> > 6 6 [->FC5base(RHFC5)] 0 1 [->admin] 2006-10-02 14:13:26 0 NULL 1 NULL
> > fc5image-FC5baseRHFC56-v0
> > 7 7 [->No Apps (WinXP vmware)] 0 1 [->admin] 2007-04-03 11:24:12 0 NULL 1
> > NULL vmwarewinxp-base7-v0
> > 8 8 [->VMware server host (RHEL4)] 0 1 [->admin] 2007-01-30 10:50:56 0 NULL
> > 1 NULL rh4image-VMwareserverhostRHEL48-v0
> > 9 9 [->VMware ESX 3.5 standard server] 0 1 [->admin] 2008-03-24 14:23:54 0
> > NULL 1 NULL esx35-base-v0
> > 10 7 [->No Apps (WinXP vmware)] 1 1 [->admin] 2010-03-18 14:56:50 0 NULL 1
> > NULL vmwarewinxp-base7-v1
> > 11 10 [->No Apps (ESXi WinXP)] 0 1 [->admin] 2010-03-24 17:35:12 0 NULL 1
> > NULL esx3-basewinxp-v0
> > /End ImageRevision Table====================
> >
> >
> > Computer Table: Rows: 3
> > ============================================
> > id stateid ownerid platformid scheduleid currentimageid
> > preferredimageid nextimageid imagerevisionid RAM procnumber procspeed
> > network hostname IPaddress privateIPaddress eth0macaddress
> > eth1macaddress type provisioningid drivetype deleted notes lastcheck
> > location dsa dsapub rsa rsapub host hostpub vmhostid vmtypeid
> > 1 10 [->maintenance] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0
> > 0 0 512 2 2200 1000 resxi-2.odu.edu<http://resxi-2.odu.edu> 128.82.128.218 NULL NULL NULL
> > virtualmachine 4 [->vmware_esx] hda 1 NULL NULL NULL NULL NULL NULL
> NULL
> > [BLOB - 0B] NULL NULL NULL
> > 2 2 [->available] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0
> > 0 20480 8 2200 1000 esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu> 128.82.128.252 NULL NULL
> > NULL blade 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL [BLOB
> -
> > 0B] NULL NULL NULL
> > 3 5 [->failed] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0 0
> > 512 2 2200 100 esx3-basewinxp-v0.odu.edu<http://esx3-basewinxp-v0.odu.edu> 128.82.128.217 NULL NULL NULL
> > virtualmachine 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL
> [BLOB
> > - 0B] NULL 1 NULL
> > /End Computer Table======================================
> >
> >
> > Vmprofile Table: Rows: 4
> > ========================
> > id profilename vmtypeid imageid nasshare datastorepath vmpath
> > virtualswitch0 virtualswitch1 vmdisk username password
> > 1 VMware GSX standard 3 [->vmwareGSX] 8 [->VMware server host (RHEL4)] NULL
> > /var/lib/vmware/Virtual Machines NULL VMnet0 VMnet2 localdisk NULL NULL
> > 2 Vmware ESX standard network mounted share 5 [->vmwareESX3] 9 [->VMware
> > ESX 3.5 standard server] NULL /vmfs/volumes/nfs1 /vmfs/volumes/storage1 VM
> > Network Virtual Machine Public Network networkdisk NULL NULL
> > 3 Vmware ESX standard localdisk 6 [->vmwareESXi] 9 [->VMware ESX 3.5
> > standard server] NULL 192.168.129.37:/install/image/vmware_images
> > 192.168.129.37:/install/image/vmware_images VM Network Private Vlan328
> > localdisk root
> > 4 Vmware ESX SAN 5 [->vmwareESX3] 9 [->VMware ESX 3.5 standard server]
> NULL
> > /vmfs/volumes/NetApp /vmfs/volumes/storage1 Intranet2 MCNC Public
> > networkdisk NULL NULL
> > /End Vmprofile===============================
> >
> > VMhost Table: Rows: 1
> > =====================
> > id computerid vmlimit vmprofileid vmkernalnic vmwaredisk
> > 1 2 [->esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu>] 2 3 NULL localdisk
> > /End Vmhost Table==========================
> >
> >
> > ================================= LOG FILE
> > ===========================================
> > 13 2010-04-01 15:37:19|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:19
> > 14 2010-04-01 15:37:24|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:24
> > 15 2010-04-01
> 15:37:24|5754|9:9|deleted|utils.pm:reservation_being_processed(9634)|computerloadlog
> > 'begin' entry does NOT exist for reservation 9
> > 16 2010-04-01
> 15:37:24|5754|9:9|deleted|utils.pm:reservation_being_processed(9686)|reservation
> > is NOT currently being processed
> > 17 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(276)|reservation 9 is
> > NOT already being processed
> > 18
> > 19 |5754|9:9|deleted| ---- WARNING ----
> > 20 |5754|9:9|deleted| 2010-04-01
> 15:37:24|5754|9:9|deleted|utils.pm:get_request_info(5218)|preferredimageid
> > is not set for computer id=2
> > 21 |5754|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 22 |5754|9:9|deleted| (-1) utils.pm<http://utils.pm>, get_request_info (line: 5218)
> > 23 |5754|9:9|deleted| (-2) vcld, main (line: 280)
> > 24
> > 25 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:get_request_info(5354)|standalone
> > affiliation found: Local
> > 26 2010-04-01
> 15:37:24|5754|9:9|deleted|utils.pm:get_management_node_info(6815)|management
> > node info retrieved from database for colbert
> > 27 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(281)|retrieved
> > request information from database
> > 28 2010-04-01
> > 15:37:24|5754|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 9
> > 29 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:update_request_state(2186)|request
> > 9 state updated to: pending, laststate to: deleted
> > 30 2010-04-01
> 15:37:24|5754|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 31 2010-04-01
> > 15:37:24|5754|9:9|deleted|vcld:make_new_child(489)|request will be processed
> > by reclaim.pm<http://reclaim.pm>
> > 32 2010-04-01 15:37:24|5754|9:9|deleted|vcld:make_new_child(509)|loaded
> > VCL::reclaim module
> > 33 2010-04-01 15:37:24|12719|9:9|deleted|vcld:make_new_child(582)|vcld
> > environment variable set to 0 for this process
> > 34 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> > called, class=VCL::reclaim
> > 35 2010-04-01
> > 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::reclaim object created
> > 36 2010-04-01
> > 15:37:24|5754|9:9|deleted|vcld:make_new_child(566)|current number of forked
> > kids: 1
> > 37 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(85)|obtained a database
> > handle for this state process, stored as $ENV{dbh}
> > 38 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:check_image_os(820)|no corrections need
> > to be made, not an imaging request, returning 1
> > 39 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 9
> > 40 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7863)|reservation
> > count: 1
> > 41 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7882)|PARENTIMAGE:
> > 1
> > 42 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7883)|SUBIMAGE:
> > 0
> > 43 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7901)|renamed
> > process to 'vcld VCL::reclaim 9:9 deleted'
> > 44 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 9
> > 45 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 9
> > 46 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(121)|attempting to load
> > provisioning module: VCL::Module::Provisioning::esx
> > 47 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(138)|esx
> > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > 48 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(140)|vmware
> > ESX module initialized
> > 49 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(127)|VCL::Module::Provisioning::esx
> > module loaded
> > 50 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> > called, class=VCL::Module::Provisioning::esx
> > 51 2010-04-01
> > 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::Provisioning::esx
> > object created
> > 52 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(138)|esx
> > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > 53 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(140)|vmware
> > ESX module initialized
> > 54 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(131)|VCL::Module::Provisioning::esx
> > provisioner object created
> > 55 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(145)|attempting to load OS
> > module: VCL::Module::OS::Linux
> > 56 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(151)|VCL::Module::OS::Linux
> > module loaded
> > 57 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> > called, class=VCL::Module::OS::Linux
> > 58 2010-04-01
> > 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::OS::Linux object
> > created
> > 59 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(154)|VCL::Module::OS::Linux
> > OS object created
> > 60 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(169)|returning 1
> > 61 2010-04-01
> > 15:37:24|12719|9:9|deleted|vcld:make_new_child(591)|VCL::reclaim object
> > created and initialized
> > 62 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation
> > remote IP is not defined
> > 63 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1946)|attempting
> > to retrieve current state of computer esx35-hypervisor-1 from the database
> > 64 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > current state of computer esx35-hypervisor-1 from the database: available
> > 65 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:_automethod(697)|data structure
> > updated: $self->request_data->{reservation}{9}{computer}{state}{name}
> > 66 |12719|9:9|deleted| computer_state_name = available
> > 67 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 68 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 69 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 70 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 71 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 72 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(138)|computer
> > type is blade
> > 73 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(142)|request
> > laststate is reserved, attempting to sanitize computer
> > 74 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(172)|calling
> > VCL::Module::OS::Linux::sanitize() subroutine
> > 75 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation
> > remote IP is not defined
> > 76 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 77 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'netstat -an' 2>&1
> > 78
> > 79 |12719|9:9|deleted| ---- WARNING ----
> > 80 |12719|9:9|deleted| 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
> > status: 255, SSH exits with the exit status of the remote command or with
> > 255 if an error occurred, output:
> > 81 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> > found
> > 82 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 83 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 84 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > 85 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > 86 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 87 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 88 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 89
> > 90 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 91 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 92 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'netstat -an' 2>&1
> > 93
> > 94 |12719|9:9|deleted| ---- WARNING ----
> > 95 |12719|9:9|deleted| 2010-04-01
> 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
> > status: 255, SSH exits with the exit status of the remote command or with
> > 255 if an error occurred, output:
> > 96 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> > found
> > 97 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 98 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 99 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > 100 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > 101 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 102 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 103 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 104
> > 105 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 106 2010-04-01 15:37:29|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:29
> > 107 2010-04-01 15:37:34|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:34
> > 108 2010-04-01 15:37:39|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:39
> > 109 2010-04-01
> 15:37:39|5754|10:10|new|utils.pm:reservation_being_processed(9634)|computerloadlog
> > 'begin' entry does NOT exist for reservation 10
> > 110 2010-04-01
> 15:37:39|5754|10:10|new|utils.pm:reservation_being_processed(9686)|reservation
> > is NOT currently being processed
> > 111 2010-04-01 15:37:39|5754|10:10|new|vcld:main(276)|reservation 10 is
> > NOT already being processed
> > 112
> > 113 |5754|10:10|new| ---- WARNING ----
> > 114 |5754|10:10|new| 2010-04-01
> 15:37:39|5754|10:10|new|utils.pm:get_request_info(5218)|preferredimageid
> > is not set for computer id=3
> > 115 |5754|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 116 |5754|10:10|new| (-1) utils.pm<http://utils.pm>, get_request_info (line: 5218)
> > 117 |5754|10:10|new| (-2) vcld, main (line: 280)
> > 118
> > 119 2010-04-01 15:37:39|5754|10:10|new|utils.pm:get_request_info(5354)|standalone
> > affiliation found: Local
> > 120 2010-04-01
> 15:37:39|5754|10:10|new|utils.pm:get_management_node_info(6815)|management
> > node info retrieved from database for colbert
> > 121 2010-04-01 15:37:39|5754|10:10|new|vcld:main(281)|retrieved request
> > information from database
> > 122 2010-04-01
> > 15:37:39|5754|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 123 2010-04-01 15:37:39|5754|10:10|new|utils.pm:update_request_state(2186)|request
> > 10 state updated to: pending, laststate to: new
> > 124 2010-04-01 15:37:39|5754|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, begin, beginning to process, state is new
> > 125 2010-04-01 15:37:39|5754|10:10|new|vcld:make_new_child(509)|loaded
> > VCL::new module
> > 126 2010-04-01 15:37:39|12726|10:10|new|vcld:make_new_child(582)|vcld
> > environment variable set to 0 for this process
> > 127 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> > called, class=VCL::new
> > 128 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::new
> > object created
> > 129 2010-04-01 15:37:39|5754|10:10|new|vcld:make_new_child(566)|current
> > number of forked kids: 2
> > 130 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(85)|obtained
> > a database handle for this state process, stored as $ENV{dbh}
> > 131 2010-04-01 15:37:39|12726|10:10|new|State.pm:check_image_os(820)|no
> > corrections need to be made, not an imaging request, returning 1
> > 132 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 133 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7863)|reservation
> > count: 1
> > 134 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7882)|PARENTIMAGE:
> > 1
> > 135 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7883)|SUBIMAGE:
> > 0
> > 136 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7901)|renamed
> > process to 'vcld VCL::new 10:10 new'
> > 137 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 138 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 139 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(121)|attempting to load
> > provisioning module: VCL::Module::Provisioning::esx
> > 140 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx
> > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > 141 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(140)|vmware
> > ESX module initialized
> > 142 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(127)|VCL::Module::Provisioning::esx
> > module loaded
> > 143 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> > called, class=VCL::Module::Provisioning::esx
> > 144 2010-04-01
> > 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::Provisioning::esx
> > object created
> > 145 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx
> > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > 146 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(140)|vmware
> > ESX module initialized
> > 147 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(131)|VCL::Module::Provisioning::esx
> > provisioner object created
> > 148 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(145)|attempting to load OS
> > module: VCL::Module::OS::Linux
> > 149 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(151)|VCL::Module::OS::Linux
> > module loaded
> > 150 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> > called, class=VCL::Module::OS::Linux
> > 151 2010-04-01
> > 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::OS::Linux object
> > created
> > 152 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(154)|VCL::Module::OS::Linux OS
> > object created
> > 153 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(169)|returning 1
> > 154 2010-04-01
> > 15:37:39|12726|10:10|new|vcld:make_new_child(591)|VCL::new object created
> > and initialized
> > 155 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 156 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > to retrieve current state of computer esx3-basewinxp-v0 from the database
> > 157 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > current state of computer esx3-basewinxp-v0 from the database: available
> > 158 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > 159 |12726|10:10|new| computer_state_name = available
> > 160
> > 161 |12726|10:10|new| ---- WARNING ----
> > 162 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_id:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> > 163 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 164 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 165 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 166 |12726|10:10|new| (-3) new.pm<http://new.pm>, process (line: 106)
> > 167 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> > 168 |12726|10:10|new| (-5) vcld, main (line: 341)
> > 169
> > 170
> > 171 |12726|10:10|new| ---- WARNING ----
> > 172 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_name:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> > 173 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 174 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 175 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 176 |12726|10:10|new| (-3) new.pm<http://new.pm>, process (line: 107)
> > 177 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> > 178 |12726|10:10|new| (-5) vcld, main (line: 341)
> > 179
> > 180 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
> > variable: 1
> > 181 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(134)|reservation
> > is parent = 1
> > 182 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(135)|preload
> > only = 0
> > 183 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(136)|originating
> > request state = new
> > 184 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(137)|originating
> > request laststate = new
> > 185 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(138)|originating
> > computer state = available
> > 186 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(139)|originating
> > computer type = virtualmachine
> > 187 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 188 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > to retrieve current state of computer esx3-basewinxp-v0 from the database
> > 189 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > current state of computer esx3-basewinxp-v0 from the database: available
> > 190 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > 191 |12726|10:10|new| computer_state_name = available
> > 192
> > 193 |12726|10:10|new| ---- WARNING ----
> > 194 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_id:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> > 195 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 196 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 197 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 198 |12726|10:10|new| (-3) new.pm<http://new.pm>, computer_not_being_used (line: 750)
> > 199 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line: 170)
> > 200 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 201 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 202
> > 203
> > 204 |12726|10:10|new| ---- WARNING ----
> > 205 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_name:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> > 206 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 207 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 208 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 209 |12726|10:10|new| (-3) new.pm<http://new.pm>, computer_not_being_used (line: 751)
> > 210 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line: 170)
> > 211 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 212 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 213
> > 214 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
> > variable: 1
> > 215 2010-04-01
> 15:37:39|12726|10:10|new|new.pm:computer_not_being_used(788)|esx3-basewinxp-v0
> > state is available
> > 216 2010-04-01
> 15:37:39|12726|10:10|new|new.pm:computer_not_being_used(798)|esx3-basewinxp-v0
> > is available, its state is available
> > 217 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(170)|esx3-basewinxp-v0
> > is not being used
> > 218 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 219 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > to retrieve current state of computer esx3-basewinxp-v0 from the database
> > 220 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > current state of computer esx3-basewinxp-v0 from the database: available
> > 221 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > 222 |12726|10:10|new| computer_state_name = available
> > 223
> > 224 |12726|10:10|new| ---- WARNING ----
> > 225 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_id:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> > 226 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 227 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 228 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 229 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image (line: 480)
> > 230 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line: 266)
> > 231 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 232 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 233
> > 234
> > 235 |12726|10:10|new| ---- WARNING ----
> > 236 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_name:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> > 237 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 238 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 239 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 240 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image (line: 481)
> > 241 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line: 266)
> > 242 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 243 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 244
> > 245 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
> > variable: 1
> > 246 2010-04-01 15:37:39|12726|10:10|new|new.pm:reload_image(514)|calling
> > VCL::Module::Provisioning::esx->node_status()
> > 247 2010-04-01 15:37:39|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, statuscheck, checking status of node
> > 248 $VAR1 = bless( {
> > 249 'data' => bless( do{\(my $o = 1)},
> > 'VCL::DataStructure' ),
> > 250 'os' => bless( {
> > 251 'provisioner' => $VAR1,
> > 252 'data' => $VAR1->{'data'}
> > 253 }, 'VCL::Module::OS::Linux' )
> > 254 }, 'VCL::Module::Provisioning::esx' );
> > 255 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(787)|Entering
> > node_status, checking status of esx3-basewinxp-v0
> > 256 2010-04-01
> 15:37:39|12726|10:10|new|esx.pm:node_status(788)|request_for_imaging:
> > 0
> > 257 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(789)|requeseted
> > image name: esx3-basewinxp-v0
> > 258 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(819)|checking
> > if esx3-basewinxp-v0 is pingable
> > 259 2010-04-01 15:37:40|12726|10:10|new|esx.pm:node_status(825)|esx3-basewinxp-
> v0
> > is not pingable (0)
> > 260 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(560)|node_status
> > returned a scalar: RELOAD
> > 261 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(602)|node
> > status is RELOAD, esx3-basewinxp-v0 will be reloaded
> > 262 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, loadimageblade, esx3-basewinxp-v0 must be reloaded with
> > esx3-basewinxp-v0
> > 263 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(616)|calling
> > VCL::Module::Provisioning::esx->does_image_exist()
> > 264 2010-04-01
> 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37<http://192.168.129.37>:
> > 265 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'ls -1 /install/image/vmware_images/golden 2>&1' 2>&1
> > 266 2010-04-01
> 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 267 |12726|10:10|new| esx3-basewinxp-v0
> > 268 2010-04-01 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "esx3-basewinxp-v0")
> > 269 2010-04-01 15:37:40|12726|10:10|new|esx.pm:does_image_exist(917)|image
> > esx3-basewinxp-v0 exists
> > 270 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(619)|esx3-
> basewinxp-v0
> > exists on this management node
> > 271 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, doesimageexists, confirmed image exists
> > 272 2010-04-01
> 15:37:40|12726|10:10|new|utils.pm:update_computer_state(2228)|computer
> > 3 state updated to: reloading
> > 273 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(652)|computer
> > esx3-basewinxp-v0 state set to reloading
> > 274 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, info, computer state updated to reloading
> > 275 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(663)|calling
> > VCL::Module::Provisioning::esx->load() subroutine
> > 276 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, info, calling VCL::Module::Provisioning::esx->load() subroutine
> > 277 2010-04-01 15:37:40|12726|10:10|new|esx.pm<http://esx.pm>:
> > load(165)|****************************************************
> > 278
> > 279 |12726|10:10|new| ---- WARNING ----
> > 280 |12726|10:10|new| 2010-04-01
> > 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_eth0_mac_address:
> > $self->request_data->{reservation}{10}{computer}{eth0macaddress}
> > 281 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 282 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 283 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 284 |12726|10:10|new| (-3) esx.pm<http://esx.pm>, load (line: 181)
> > 285 |12726|10:10|new| (-4) new.pm<http://new.pm>, reload_image (line: 665)
> > 286 |12726|10:10|new| (-5) new.pm<http://new.pm>, process (line: 266)
> > 287 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
> > 288
> > 289
> > 290 |12726|10:10|new| ---- WARNING ----
> > 291 |12726|10:10|new| 2010-04-01
> > 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_eth1_mac_address:
> > $self->request_data->{reservation}{10}{computer}{eth1macaddress}
> > 292 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 293 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 294 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 295 |12726|10:10|new| (-3) esx.pm<http://esx.pm>, load (line: 182)
> > 296 |12726|10:10|new| (-4) new.pm<http://new.pm>, reload_image (line: 665)
> > 297 |12726|10:10|new| (-5) new.pm<http://new.pm>, process (line: 266)
> > 298 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
> > 299
> > 300 2010-04-01 15:37:40|12726|10:10|new|esx.pm:load(193)|Calling
> > get_vmware_host_info
> > 301 No recipient addresses found in header
> > 302 2010-04-01 15:37:41|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
> > Sending mail To: , PROBLEM -- esx.pm<http://esx.pm>
> > 303
> > 304 |12726|10:10|new| ---- CRITICAL ----
> > 305 |12726|10:10|new| 2010-04-01
> 15:37:40|12726|10:10|new|esx.pm:get_vmware_host_info(1012)|AMIT
> > DEBUGGING: esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu>
> > 306 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 307 |12726|10:10|new| (-1) esx.pm<http://esx.pm>, get_vmware_host_info (line: 1012)
> > 308 |12726|10:10|new| (-2) esx.pm<http://esx.pm>, load (line: 194)
> > 309 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image (line: 665)
> > 310 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line: 266)
> > 311 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 312 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 313
> > 314 2010-04-01 15:37:41|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 315 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'netstat -an' 2>&1
> > 316
> > 317 |12719|9:9|deleted| ---- WARNING ----
> > 318 |12719|9:9|deleted| 2010-04-01
> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 3/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
> > status: 255, SSH exits with the exit status of the remote command or with
> > 255 if an error occurred, output:
> > 319 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> > found
> > 320 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 321 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 322 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > 323 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > 324 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 325 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 326 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 327
> > 328
> > 329 |12719|9:9|deleted| ---- WARNING ----
> > 330 |12719|9:9|deleted| 2010-04-01
> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'netstat -an' 2>&1,
> > exit status: 255, output:
> > 331 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> > found
> > 332 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 333 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6285)
> > 334 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > 335 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > 336 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 337 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 338 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 339
> > 340 2010-04-01
> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 341 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > /etc/ssh/external_sshd_config"' 2>&1
> > 342
> > 343 |12719|9:9|deleted| ---- WARNING ----
> > 344 |12719|9:9|deleted| 2010-04-01
> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx
> > "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255, SSH
> > exits with the exit status of the remote command or with 255 if an error
> > occurred, output:
> > 345 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not
> > found
> > 346 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 347 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 348 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line: 913)
> > 349 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
> > 350 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
> > 351 |12719|9:9|deleted| (-5) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 352 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
> > 353
> > 354 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 355 2010-04-01 15:37:43|12726|10:10|new|esx.pm:get_vmware_host_info(1019)|host
> > info output for esx35-hypervisor-1
> > 356 |12726|10:10|new| Host Information
> > 357 |12726|10:10|new| Host Name: esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu>
> > 358 2010-04-01 15:37:43|12726|10:10|new|esx.pm:get_vmware_host_info(1025)|found
> > hostname_value= esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu>
> > 359 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(197)|Collected
> > esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu> for vmware host name
> > 360 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(210)|DATASTORE IP
> > is 192.168.129.37 and DATASTORE_SHARE_PATH is /install/image/vmware_images
> > 361 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(211)|Entered ESX
> > module, loading esx3-basewinxp-v0 on esx3-basewinxp-v0 (on
> > esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu>) for reservation 10
> > 362 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(212)|Datastore:
> > 192.168.129.37:/install/image/vmware_images
> > 363 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(223)|VM info
> > command: /usr/lib/vmware-vcli/apps/vm/vminfo.pl<http://vminfo.pl> --server
> > 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --username root --password
> > 'xxxxxxxxxxx'
> > 364 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 365 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > /etc/ssh/external_sshd_config"' 2>&1
> > 366 2010-04-01 15:37:44|12726|10:10|new|esx.pm:load(226)|VM info
> > output: Virtual Machine esx3-basewinxp-v0 not found.
> > 367 2010-04-01
> 15:37:44|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37<http://192.168.129.37>:
> > 368 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'rm -rf /install/image/vmware_images/inuse/esx3-basewinxp-v0'
> > 2>&1
> > 369 2010-04-01 15:37:44|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:44
> > 370
> > 371 |12719|9:9|deleted| ---- WARNING ----
> > 372 |12719|9:9|deleted| 2010-04-01
> 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx
> > "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255, SSH
> > exits with the exit status of the remote command or with 255 if an error
> > occurred, output:
> > 373 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not
> > found
> > 374 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 375 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 376 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line: 913)
> > 377 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
> > 378 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
> > 379 |12719|9:9|deleted| (-5) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 380 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
> > 381
> > 382 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 383 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 384 |12726|10:10|new| none
> > 385 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "none")
> > 386 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(262)|Removed old vm
> > folder
> > 387 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37<http://192.168.129.37>:
> > 388 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'mkdir /install/image/vmware_images/inuse/esx3-basewinxp-v0'
> > 2>&1
> > 389 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 390 |12726|10:10|new| none
> > 391 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "none")
> > 392 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37<http://192.168.129.37>:
> > 393 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'cp
> > /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk
> > /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk'
> > 2>&1
> > 394 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 395 |12726|10:10|new| none
> > 396 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "none")
> > 397 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(278)|COPIED VMDK
> > SUCCESSFULLY
> > 398 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37<http://192.168.129.37>:
> > 399 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'cp
> > /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk
> > /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk'
> > 2>&1
> > 400 2010-04-01 15:37:49|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:49
> > 401 2010-04-01 15:37:54|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:54
> > 402 2010-04-01 15:37:59|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 403 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > /etc/ssh/external_sshd_config"' 2>&1
> > 404 2010-04-01 15:37:59|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:59
> > 405 2010-04-01
> 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 406 |12719|9:9|deleted| ash: pkill: not found
> > 407 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6272)|SSH
> > command executed on esx35-hypervisor-1, command:
> > 408 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > /etc/ssh/external_sshd_config"' 2>&1
> > 409 |12719|9:9|deleted| returning (127, "ash: pkill: not found")
> > 410 2010-04-01
> > 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(916)|output detected:
> > ash: pkill: not found
> > 411 2010-04-01
> > 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(921)|ext_sshd on
> > esx35-hypervisor-1 stopped
> > 412 2010-04-01
> > 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(884)|stopped external sshd
> > 413 2010-04-01
> > 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(888)|access has been
> > revoked to esx35-hypervisor-1
> > 414 2010-04-01
> 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 415 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 416
> > 417 |12719|9:9|deleted| ---- WARNING ----
> > 418 |12719|9:9|deleted| 2010-04-01
> 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 419 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 420 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 421 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 422 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 423 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 424 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 425 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > 426 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 427 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 428 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 429 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 430
> > 431 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 432 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 433 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 434
> > 435 |12719|9:9|deleted| ---- WARNING ----
> > 436 |12719|9:9|deleted| 2010-04-01
> 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 437 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 438 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 439 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 440 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 441 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 442 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 443 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > 444 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 445 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 446 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 447 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 448
> > 449 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 450 2010-04-01 15:38:04|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:04
> > 451 2010-04-01 15:38:09|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:09
> > 452 2010-04-01 15:38:14|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:14
> > 453 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 454 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 455
> > 456 |12719|9:9|deleted| ---- WARNING ----
> > 457 |12719|9:9|deleted| 2010-04-01
> 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 3/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 458 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 459 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 460 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 461 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 462 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 463 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 464 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > 465 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 466 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 467 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 468 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 469
> > 470
> > 471 |12719|9:9|deleted| ---- WARNING ----
> > 472 |12719|9:9|deleted| 2010-04-01
> 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel
> > admin' 2>&1, exit status: 255, output:
> > 473 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 474 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 475 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 476 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 477 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 478 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6285)
> > 479 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > 480 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 481 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 482 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 483 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 484
> > 485 2010-04-01
> 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 486 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 487
> > 488 |12719|9:9|deleted| ---- WARNING ----
> > 489 |12719|9:9|deleted| 2010-04-01
> 15:38:18|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 490 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 491 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 492 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 493 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 494 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 495 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 496 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > 497 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 498 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 499 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 500 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 501
> > 502 2010-04-01 15:38:18|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 503 2010-04-01 15:38:19|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:19
> > 504 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 505 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 506
> > 507 |12719|9:9|deleted| ---- WARNING ----
> > 508 |12719|9:9|deleted| 2010-04-01
> 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 509 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 510 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 511 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 512 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 513 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 514 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 515 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > 516 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 517 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 518 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 519 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 520
> > 521 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 522 2010-04-01 15:38:25|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:24
> > 523 2010-04-01 15:38:30|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:30
> > 524 2010-04-01 15:38:35|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:35
> > 525 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 526 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 527
> > 528 |12719|9:9|deleted| ---- WARNING ----
> > 529 |12719|9:9|deleted| 2010-04-01
> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 3/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 530 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 531 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 532 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 533 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 534 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 535 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 536 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > 537 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 538 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 539 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 540 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 541
> > 542
> > 543 |12719|9:9|deleted| ---- WARNING ----
> > 544 |12719|9:9|deleted| 2010-04-01
> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel
> > admin' 2>&1, exit status: 255, output:
> > 545 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 546 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 547 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 548 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 549 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 550 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6285)
> > 551 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > 552 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 553 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 554 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 555 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 556
> > 557 2010-04-01
> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 558 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> > /etc/ssh/external_sshd_config' 2>&1
> > 559
> > 560 |12719|9:9|deleted| ---- WARNING ----
> > 561 |12719|9:9|deleted| 2010-04-01
> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> > "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
> > exits with the exit status of the remote command or with 255 if an error
> > occurred, output:
> > 562 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > /etc/ssh/external_sshd_config: No such file or directory
> > 563 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 564 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 565 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > 566 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 567 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 568 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 569 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 570
> > 571 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 572 2010-04-01 15:38:37|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 573 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> > /etc/ssh/external_sshd_config' 2>&1
> > 574
> > 575 |12719|9:9|deleted| ---- WARNING ----
> > 576 |12719|9:9|deleted| 2010-04-01
> 15:38:38|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> > "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
> > exits with the exit status of the remote command or with 255 if an error
> > occurred, output:
> > 577 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > /etc/ssh/external_sshd_config: No such file or directory
> > 578 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 579 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 580 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > 581 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 582 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 583 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 584 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 585
> > 586 2010-04-01 15:38:38|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 587 2010-04-01 15:38:40|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:40
> > 588 2010-04-01 15:38:45|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:45
> > 589 2010-04-01 15:38:50|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:50
> > 590 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 591 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> > /etc/ssh/external_sshd_config' 2>&1
> > 592
> > 593 |12719|9:9|deleted| ---- WARNING ----
> > 594 |12719|9:9|deleted| 2010-04-01
> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> > "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
> > exits with the exit status of the remote command or with 255 if an error
> > occurred, output:
> > 595 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > /etc/ssh/external_sshd_config: No such file or directory
> > 596 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 597 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 598 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > 599 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 600 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 601 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 602 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 603
> > 604
> > 605 |12719|9:9|deleted| ---- WARNING ----
> > 606 |12719|9:9|deleted| 2010-04-01
> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^AllowUsers
> > .*/d" /etc/ssh/external_sshd_config' 2>&1, exit status: 255, output:
> > 607 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > /etc/ssh/external_sshd_config: No such file or directory
> > 608 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 609 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6285)
> > 610 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > 611 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 612 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 613 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 614 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 615
> > 616 No recipient addresses found in header
> > 617 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:mail(1348)|SUCCESS
> > -- Sending mail To: , PROBLEM -- Linux.pm
> > 618
> > 619 |12719|9:9|deleted| ---- CRITICAL ----
> > 620 |12719|9:9|deleted| 2010-04-01
> > 15:38:53|12719|9:9|deleted|Linux.pm:delete_user(566)|failed to add
> > AllowUsers admin to external_sshd_config
> > 621 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 622 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 566)
> > 623 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
> > 624 |12719|9:9|deleted| (-3) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 625 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
> > 626 |12719|9:9|deleted| (-5) vcld, main (line: 341)
> > 627
> > 628 2010-04-01
> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 629 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> > 630
> > 631 |12719|9:9|deleted| ---- WARNING ----
> > 632 |12719|9:9|deleted| 2010-04-01
> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
> > .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
> > remote command or with 255 if an error occurred, output:
> > 633 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> > No such file or directory
> > 634 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 635 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 636 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > 637 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 638 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 639 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 640 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 641
> > 642 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 643 2010-04-01 15:38:55|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:55
> > 644 2010-04-01 15:38:55|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 645 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> > 646
> > 647 |12719|9:9|deleted| ---- WARNING ----
> > 648 |12719|9:9|deleted| 2010-04-01
> 15:38:56|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
> > .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
> > remote command or with 255 if an error occurred, output:
> > 649 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> > No such file or directory
> > 650 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 651 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 652 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > 653 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 654 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 655 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 656 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 657
> > 658 2010-04-01 15:38:56|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 659 2010-04-01 15:39:00|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:00
> > 660 2010-04-01 15:39:05|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:05
> > 661 2010-04-01 15:39:10|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:10
> > 662 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 663 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> > 664
> > 665 |12719|9:9|deleted| ---- WARNING ----
> > 666 |12719|9:9|deleted| 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
> > .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
> > remote command or with 255 if an error occurred, output:
> > 667 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> > No such file or directory
> > 668 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 669 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6252)
> > 670 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > 671 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 672 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 673 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 674 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 675
> > 676
> > 677 |12719|9:9|deleted| ---- WARNING ----
> > 678 |12719|9:9|deleted| 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^admin
> > .*/d" /etc/sudoers' 2>&1, exit status: 255, output:
> > 679 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> > No such file or directory
> > 680 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 681 |12719|9:9|deleted| (-1) utils.pm<http://utils.pm>, run_ssh_command (line: 6285)
> > 682 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > 683 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 684 |12719|9:9|deleted| (-4) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 685 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 686 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 687
> > 688 No recipient addresses found in header
> > 689 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:mail(1348)|SUCCESS
> > -- Sending mail To: , PROBLEM -- Linux.pm
> > 690
> > 691 |12719|9:9|deleted| ---- CRITICAL ----
> > 692 |12719|9:9|deleted| 2010-04-01
> > 15:39:11|12719|9:9|deleted|Linux.pm:delete_user(578)|failed to clear admin
> > from /etc/sudoers
> > 693 |12719|9:9|deleted| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 694 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 578)
> > 695 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
> > 696 |12719|9:9|deleted| (-3) reclaim.pm<http://reclaim.pm>, process (line: 173)
> > 697 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
> > 698 |12719|9:9|deleted| (-5) vcld, main (line: 341)
> > 699
> > 700 2010-04-01 15:39:11|12719|9:9|deleted|Linux.pm:sanitize(852)|users
> > have been deleted from esx35-hypervisor-1
> > 701 2010-04-01 15:39:11|12719|9:9|deleted|reclaim.pm:process(173)|esx35-
> hypervisor-1
> > has been sanitized
> > 702 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8018)|called
> > from VCL::reclaim::process(271)
> > 703 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8054)|parent:
> > parent reservation ID for this request: 9
> > 704 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 705 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:update_request_state(2186)|request
> > 9 state updated to: complete, laststate to: deleted
> > 706 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8078)|req=9:
> > request state changed: deleted->complete, laststate: reserved->deleted
> > 707 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 708 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 709 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:update_computer_state(2228)|computer
> > 2 state updated to: available
> > 710 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8096)|req=9:
> > computer esx35-hypervisor-1 state changed: available->available
> > 711 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8106)|req=9:
> > log table id=10 will not be updated
> > 712 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 713 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8122)|req=9:
> > process exiting
> > 714 2010-04-01
> > 15:39:11|12719|9:9|deleted|State.pm:DESTROY(905)|destructor called,
> > ref($self)=VCL::reclaim
> > 715 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:delete_computerloadlog_reservation(7551)|removing
> > computerloadlog entries matching loadstate = begin
> > 716 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:delete_computerloadlog_reservation(7598)|deleted
> > rows from computerloadlog for reservation id=9
> > 717 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(912)|removed
> > computerloadlog rows with loadstate=begin for reservation
> > 718 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(924)|number
> > of database handles state process created: 1
> > 719 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(933)|process
> > has a database handle stored in $ENV{dbh}, attempting disconnect
> > 720 2010-04-01
> > 15:39:11|12719|9:9|deleted|State.pm:DESTROY(935)|$ENV{dbh}: database
> > disconnect successful
> > 721 2010-04-01
> > 15:39:11|12719|9:9|deleted|State.pm:DESTROY(949)|VCL::reclaim process 12719
> > exiting
> > 722 2010-04-01 15:39:11|5754|vcld:REAPER(744)|VCL process exited for
> > reservation 9
> > 723 2010-04-01 15:39:11|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:11
> > 724 2010-04-01 15:39:11|5754|9:9|complete|vcld:main(250)|request deleted
> > 725 2010-04-01 15:39:16|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:16
> > 726 2010-04-01 15:39:21|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:21
> > 727 2010-04-01 15:39:26|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:26
> > 728 2010-04-01 15:39:31|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:31
> > 729 2010-04-01 15:39:36|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:36
> > 730 2010-04-01 15:39:42|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:42
> > 731 2010-04-01 15:39:47|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:47
> > 732 2010-04-01 15:39:52|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:52
> > 733 2010-04-01
> 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 734 |12726|10:10|new| none
> > 735 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "none")
> > 736 2010-04-01
> 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37<http://192.168.129.37>:
> > 737 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'grep adapterType
> > /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk
> > 2>&1' 2>&1
> > 738 2010-04-01
> 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 739 |12726|10:10|new| ddb.adapterType = "buslogic"
> > 740 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "ddb.adapterType =
> > "buslogic"")
> > 741 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(311)|adapter=
> > buslogic
> > 742 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(342)|eth0 MAC
> > address set for vcl assigned
> > 743 Use of uninitialized value in concatenation (.) or string at
> > 744 /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pmline 343 (#1)
> > 745 (W uninitialized) An undefined value was used as if it were
> > already
> > 746 defined. It was interpreted as a "" or a 0, but maybe it was a
> > mistake.
> > 747 To suppress this warning assign a defined value to your
> > variables.
> > 748
> > 749 To help you figure out what was undefined, perl tells you what
> > operation
> > 750 you used the undefined value in. Note, however, that perl
> > optimizes your
> > 751 program and the operation displayed in the warning may not
> > necessarily
> > 752 appear literally in your program. For example, "that $foo" is
> > 753 usually optimized into "that " . $foo, and the warning will
> > refer to
> > 754 the concatenation (.) operator, even though there is no . in
> > your
> > 755 program.
> > 756
> > 757
> > 758 |12726|10:10|new| ---- WARNING ----
> > 759 |12726|10:10|new| 2010-04-01
> > 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> > value in concatenation (.) or string at
> > /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm<http://esx.pm> line 343.
> > 760 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 761 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> > 762 |12726|10:10|new| (-2) esx.pm<http://esx.pm>, load (line: 343)
> > 763 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image (line: 665)
> > 764 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line: 266)
> > 765 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 766 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 767
> > 768
> > 769 |12726|10:10|new| ---- WARNING ----
> > 770 |12726|10:10|new| 2010-04-01
> > 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> > value in concatenation (.) or string at
> > /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm<http://esx.pm> line 353.
> > 771 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 772 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> > 773 |12726|10:10|new| (-2) esx.pm<http://esx.pm>, load (line: 353)
> > 774 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image (line: 665)
> > 775 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line: 266)
> > 776 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 777 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 778
> > 779 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(353)|eth1 MAC
> > address set for vcl assigned
> > 780
> > 781 |12726|10:10|new| ---- WARNING ----
> > 782 |12726|10:10|new| 2010-04-01
> > 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> > value in concatenation (.) or string at
> > /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm<http://esx.pm> line 354.
> > 783 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 784 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> > 785 |12726|10:10|new| (-2) esx.pm<http://esx.pm>, load (line: 354)
> > 786 |12726|10:10|new| (-3) new.pm<http://new.pm>, reload_image (line: 665)
> > 787 |12726|10:10|new| (-4) new.pm<http://new.pm>, process (line: 266)
> > 788 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 789 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 790
> > 791 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(376)|wrote vmxarray
> > to /tmp/esx3-basewinxp-v0.vmx
> > 792 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_scp_command(6403)|attempt
> > 1/3: executing SCP command: /usr/bin/scp -B -i /etc/vcl/vcl.key -P 22 -p -r
> > /tmp/esx3-basewinxp-v0.vmx 192.168.129.37:/install/image/vmware_images/inuse/esx3-
> basewinxp-v0/esx3-basewinxp-v0.vmx
> > 2>&1
> > 793 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_scp_command(6453)|scp
> > successful: attempt 1/3, exit status: 0, output: none
> > 794 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(401)|Register
> > Command: /usr/lib/vmware-vcli/apps/vm/vmregister.pl<http://vmregister.pl> --server
> > 'esx35-hypervisor-1' --username root --password 'xxxxxxxxxxx' --vmxpath
> > '[VCL]/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmx' --operation register
> > --vmname esx3-basewinxp-v0 --pool Resources --hostname '
> > esx35-hypervisor-1.odu.edu<http://esx35-hypervisor-1.odu.edu>' --datacenter 'ha-datacenter'
> > 795 2010-04-01 15:39:57|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:57
> > 796 2010-04-01 15:39:57|12726|10:10|new|esx.pm:load(404)|Registered:
> > 797 |12726|10:10|new| SOAP Fault:
> > 798 |12726|10:10|new| -----------
> > 799 |12726|10:10|new| Fault string: fault.RestrictedVersion.summary
> > 800 |12726|10:10|new| Fault detail: RestrictedVersionFault
> > 801 2010-04-01 15:39:57|12726|10:10|new|esx.pm:load(413)|Power on
> > command: /usr/lib/vmware-vcli/apps/vm/vmcontrol.pl<http://vmcontrol.pl> --server
> > 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --operation poweron
> > --username root --password 'xxxxxxxxxxx'
> > 802 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(416)|Powered on:
> > Virtual Machine esx3-basewinxp-v0 not found.
> > 803 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(505)|IP is known
> > for esx3-basewinxp-v0
> > 804 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(511)|Waiting for
> > ssh to come up on esx3-basewinxp-v0
> > 805 2010-04-01 15:40:00|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 1
> > 806 2010-04-01 15:40:02|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:02
> > 807 2010-04-01 15:40:06|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 2
> > 808 2010-04-01 15:40:07|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:07
> > 809 2010-04-01 15:40:12|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:12
> > 810 2010-04-01 15:40:12|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 3
> > 811 2010-04-01 15:40:17|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:17
> > 812 2010-04-01 15:40:18|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 4
> > 813 2010-04-01 15:40:22|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:22
> > 814 2010-04-01 15:40:24|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 5
> > 815 2010-04-01 15:40:27|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:27
> > 816 2010-04-01 15:40:30|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 6
> > 817 2010-04-01 15:40:32|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:32
> > 818 2010-04-01 15:40:36|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 7
> > 819 2010-04-01 15:40:37|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:37
> > 820 2010-04-01 15:40:42|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:42
> > 821 2010-04-01 15:40:42|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 8
> > 822 2010-04-01 15:40:47|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:47
> > 823 2010-04-01 15:40:48|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 9
> > 824 2010-04-01 15:40:52|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:52
> > 825 2010-04-01 15:40:54|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 10
> > 826 2010-04-01 15:40:57|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:57
> > 827 2010-04-01 15:41:00|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 11
> > 828 2010-04-01 15:41:02|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:02
> > 829 2010-04-01 15:41:06|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 12
> > 830 2010-04-01 15:41:07|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:07
> > 831 2010-04-01 15:41:12|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:12
> > 832 2010-04-01 15:41:12|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 13
> > 833 2010-04-01 15:41:17|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:17
> > 834 2010-04-01 15:41:18|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 14
> > 835 2010-04-01 15:41:22|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:22
> > 836 2010-04-01 15:41:24|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 15
> > 837 2010-04-01 15:41:27|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:27
> > 838 2010-04-01 15:41:30|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 16
> > 839 2010-04-01 15:41:32|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:32
> > 840 2010-04-01 15:41:36|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 17
> > 841 2010-04-01 15:41:37|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:37
> > 842 2010-04-01 15:41:42|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:42
> > 843 2010-04-01 15:41:42|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 18
> > 844 2010-04-01 15:41:47|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:47
> > 845 2010-04-01 15:41:48|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 19
> > 846 2010-04-01 15:41:52|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:52
> > 847 2010-04-01 15:41:54|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 20
> > 848 2010-04-01 15:41:57|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:57
> > 849 2010-04-01 15:42:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 21
> > 850 2010-04-01 15:42:02|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:02
> > 851 2010-04-01 15:42:07|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 22
> > 852 2010-04-01 15:42:07|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:07
> > 853 2010-04-01 15:42:12|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:12
> > 854 2010-04-01 15:42:13|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 23
> > 855 2010-04-01 15:42:18|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:18
> > 856 2010-04-01 15:42:19|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 24
> > 857 2010-04-01 15:42:23|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:23
> > 858 2010-04-01 15:42:25|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 25
> > 859 2010-04-01 15:42:28|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:28
> > 860 2010-04-01 15:42:31|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 26
> > 861 2010-04-01 15:42:33|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:33
> > 862 2010-04-01 15:42:37|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 27
> > 863 2010-04-01 15:42:38|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:38
> > 864 2010-04-01 15:42:43|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 28
> > 865 2010-04-01 15:42:43|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:43
> > 866 2010-04-01 15:42:48|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:48
> > 867 2010-04-01 15:42:49|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 29
> > 868 2010-04-01 15:42:53|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:53
> > 869 2010-04-01 15:42:55|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 30
> > 870 2010-04-01 15:42:58|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:58
> > 871 2010-04-01 15:43:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 31
> > 872 2010-04-01 15:43:03|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:03
> > 873 2010-04-01 15:43:07|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 32
> > 874 2010-04-01 15:43:08|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:08
> > 875 2010-04-01 15:43:13|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 33
> > 876 2010-04-01 15:43:13|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:13
> > 877 2010-04-01 15:43:18|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:18
> > 878 2010-04-01 15:43:19|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 34
> > 879 2010-04-01 15:43:23|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:23
> > 880 2010-04-01 15:43:25|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 35
> > 881 2010-04-01 15:43:28|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:28
> > 882 2010-04-01 15:43:31|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 36
> > 883 2010-04-01 15:43:33|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:33
> > 884 2010-04-01 15:43:37|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 37
> > 885 2010-04-01 15:43:38|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:38
> > 886 2010-04-01 15:43:43|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 38
> > 887 2010-04-01 15:43:43|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:43
> > 888 2010-04-01 15:43:48|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:48
> > 889 2010-04-01 15:43:49|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 39
> > 890 2010-04-01 15:43:53|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:53
> > 891 2010-04-01 15:43:55|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 40
> > 892 2010-04-01 15:43:58|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:58
> > 893 2010-04-01 15:44:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 41
> > 894 2010-04-01 15:44:03|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:03
> > 895 2010-04-01 15:44:07|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 42
> > 896 2010-04-01 15:44:08|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:08
> > 897 2010-04-01 15:44:13|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 43
> > 898 2010-04-01 15:44:13|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:13
> > 899 2010-04-01 15:44:18|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:18
> > 900 2010-04-01 15:44:19|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 44
> > 901 2010-04-01 15:44:23|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:23
> > 902 2010-04-01 15:44:25|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 45
> > 903 2010-04-01 15:44:29|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:28
> > 904 2010-04-01 15:44:31|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 46
> > 905 2010-04-01 15:44:34|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:34
> > 906 2010-04-01 15:44:37|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 47
> > 907 2010-04-01 15:44:39|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:39
> > 908 2010-04-01 15:44:43|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 48
> > 909 2010-04-01 15:44:44|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:44
> > 910 2010-04-01 15:44:49|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:49
> > 911 2010-04-01 15:44:49|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 49
> > 912 2010-04-01 15:44:54|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:54
> > 913 2010-04-01 15:44:55|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 50
> > 914 2010-04-01 15:44:59|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:59
> > 915 2010-04-01 15:45:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 51
> > 916 2010-04-01 15:45:04|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:45:04
> > 917 No recipient addresses found in header
> > 918 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
> > Sending mail To: , PROBLEM -- esx.pm<http://esx.pm>
> > 919
> > 920 |12726|10:10|new| ---- CRITICAL ----
> > 921 |12726|10:10|new| 2010-04-01 15:45:07|12726|10:10|new|esx.pm:load(521)|waited
> > acceptable amount of time for sshd to become active, please check
> > esx3-basewinxp-v0 on esx35-hypervisor-1
> > 922 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 923 |12726|10:10|new| (-1) esx.pm<http://esx.pm>, load (line: 521)
> > 924 |12726|10:10|new| (-2) new.pm<http://new.pm>, reload_image (line: 665)
> > 925 |12726|10:10|new| (-3) new.pm<http://new.pm>, process (line: 266)
> > 926 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> > 927 |12726|10:10|new| (-5) vcld, main (line: 341)
> > 928
> > 929
> > 930 |12726|10:10|new| ---- WARNING ----
> > 931 |12726|10:10|new| 2010-04-01
> 15:45:08|12726|10:10|new|new.pm:reload_image(670)|esx3-basewinxp-v0
> > failed to load on esx3-basewinxp-v0, returning
> > 932 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 933 |12726|10:10|new| (-1) new.pm<http://new.pm>, reload_image (line: 670)
> > 934 |12726|10:10|new| (-2) new.pm<http://new.pm>, process (line: 266)
> > 935 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
> > 936 |12726|10:10|new| (-4) vcld, main (line: 341)
> > 937
> > 938 2010-04-01 15:45:08|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, loadimagefailed, esx3-basewinxp-v0 failed to load on
> > esx3-basewinxp-v0
> > 939
> > 940 |12726|10:10|new| ---- WARNING ----
> > 941 |12726|10:10|new| 2010-04-01
> 15:45:08|12726|10:10|new|new.pm:process(313)|failed
> > to load esx3-basewinxp-v0 with esx3-basewinxp-v0
> > 942 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 943 |12726|10:10|new| (-1) new.pm<http://new.pm>, process (line: 313)
> > 944 |12726|10:10|new| (-2) vcld, make_new_child (line: 594)
> > 945 |12726|10:10|new| (-3) vcld, main (line: 341)
> > 946
> > 947 2010-04-01
> > 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > to retrieve current state of computer esx3-basewinxp-v0 from the database
> > 948 2010-04-01
> > 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > current state of computer esx3-basewinxp-v0 from the database: reloading
> > 949 2010-04-01
> > 15:45:08|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > 950 |12726|10:10|new| computer_state_name = reloading
> > 951 No recipient addresses found in header
> > 952 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
> > Sending mail To: , PROBLEM -- State.pm
> > 953
> > 954 |12726|10:10|new| ---- CRITICAL ----
> > 955 |12726|10:10|new| 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(290)|reservation failed
> > on esx3-basewinxp-v0: process failed after trying to load or make available
> > 956 |12726|10:10|new| ( 0) utils.pm<http://utils.pm>, notify (line: 737)
> > 957 |12726|10:10|new| (-1) State.pm, reservation_failed (line: 290)
> > 958 |12726|10:10|new| (-2) new.pm<http://new.pm>, process (line: 316)
> > 959 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
> > 960 |12726|10:10|new| (-4) vcld, main (line: 341)
> > 961
> > 962 2010-04-01 15:45:08|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, failed, process failed after trying to load or make available
> > 963 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(293)|inserted
> > computerloadlog entry
> > 964 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(301)|updated log ending
> > value to 'failed', logid=11
> > 965 2010-04-01
> 15:45:08|12726|10:10|new|utils.pm:update_computer_state(2228)|computer
> > 3 state updated to: failed
> > 966 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(312)|computer
> > esx3-basewinxp-v0 (3) state set to failed
> > 967 2010-04-01 15:45:08|12726|10:10|new|utils.pm:update_request_state(2186)|request
> > 10 state updated to: failed, laststate to: new
> > 968 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(325)|set request state
> > to 'failed'/'new'
> > 969 2010-04-01 15:45:08|12726|10:10|new|utils.pm:is_inblockrequest(6972)|zero
> > rows were returned from database select
> > 970 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(343)|esx3-basewinxp-v0
> > is NOT in blockcomputers table
> > 971 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(346)|exiting 1
> > 972 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(905)|destructor
> > called, ref($self)=VCL::new
> > 973 2010-04-01
> 15:45:08|12726|10:10|new|utils.pm:delete_computerloadlog_reservation(7551)|removing
> > computerloadlog entries matching loadstate = begin
> > 974 2010-04-01
> 15:45:08|12726|10:10|new|utils.pm:delete_computerloadlog_reservation(7598)|deleted
> > rows from computerloadlog for reservation id=10
> > 975 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(912)|removed
> > computerloadlog rows with loadstate=begin for reservation
> > 976 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(924)|number of
> > database handles state process created: 1
> > 977 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(933)|process
> > has a database handle stored in $ENV{dbh}, attempting disconnect
> > 978 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(935)|$ENV{dbh}:
> > database disconnect successful
> > 979 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(949)|VCL::new
> > process 12726 exiting
> > 980 2010-04-01 15:45:08|5754|vcld:REAPER(744)|VCL process exited for
> > reservation 10
> > 981 2010-04-01 15:45:08|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:45:08
> > 982 2010-04-01 15:45:13|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:45:13
> >
> >
> ================================================================
> =====================
> >
> > Thank you,
> > Amit
> >
> >
> >
RE: [Spam: 6.0] Re: [Spam: 6.0] Re: VCL Getting Reservation to work
Posted by "Kumar, Amit H." <AH...@odu.edu>.
Attached are the screen shots.
Thank you,
Amit
> -----Original Message-----
> From: Ryan Johnson [mailto:rjohn06@gwmail.gwu.edu]
> Sent: Thursday, April 01, 2010 4:23 PM
> To: vcl-dev@incubator.apache.org
> Subject: [Spam: 6.0] Re: [Spam: 6.0] Re: VCL Getting Reservation to work
>
> Yea, that is a little hard on the eyes, any chance you could send screen
> shots in a view where the columns are lined up with the column names?
>
> On Thu, Apr 1, 2010 at 4:00 PM, Kumar, Amit H. <AH...@odu.edu> wrote:
>
> > The Savior! Thank you Ryan getting back!!!
> >
> > Not sure if this is the best format for readability...please let me know
> > otherwise.
> >
> > I am also a recent log that I tried with verbose ssh enabled etc, I haven't
> > looked at it yet. It went past my previous log. .....................
> >
> > Image: Table: Rows: 10
> > ====================
> > id name prettyname ownerid platformid OSid imagemetaid minram
> > minprocnumber minprocspeed minnetwork maxconcurrent reloadtime deleted
> > test lastupdate forcheckout maxinitialtime project size architecture
> > description usage basedoffrevisionid
> > 1 winxp-base1-v0 No Apps (WinXP) 1 [->admin] 1 [->i386] 7 [->winxp] NULL 0
> > 1 0 10 NULL 14 0 0 2007-04-11 16:07:38 1 0 vcl 1045 x86 NULL NULL 0
> > 2 rhel4-base2-v0 Red Hat Enterprise Linux 4.4 Base (KS) 1 [->admin] 1
> > [->i386] 12 [->rhel4] NULL 1024 1 1024 100 NULL 13 0 0 2007-03-02 16:33:33 1
> > 0 vcl 0 x86 NULL NULL 0
> > 3 rh4image-base3-v0 RHEL4 base (image) 1 [->admin] 1 [->i386] 13
> > [->rh4image] NULL 10 1 1024 100 NULL 10 0 0 2007-01-24 15:02:07 1 0 vcl 1450
> > x86 NULL NULL 0
> > 4 noimage No Image 1 [->admin] 1 [->i386] 2 [->sun4x_58] NULL 0 1 0 10 NULL
> > 0 0 0 NULL 1 0 vcl 1450 x86 NULL NULL 0
> > 5 rhfc5-fc5base5-v0 Red Hat Fedora Core 5 base (KS) 1 [->admin] 1 [->i386]
> > 15 [->rhfc5] NULL 1024 1 1024 100 NULL 13 0 0 2006-10-02 10:04:24 1 0 vcl
> > 1450 x86 NULL NULL 0
> > 6 fc5image-FC5baseRHFC56-v0 FC5base(RHFC5) 1 [->admin] 1 [->i386] 14
> > [->fc5image] NULL 64 1 500 10 NULL 10 0 0 2006-10-02 14:13:26 1 0 vcl 0 x86
> > NULL NULL 0
> > 7 vmwarewinxp-base7-v1 No Apps (WinXP vmware) 1 [->admin] 1 [->i386] 16
> > [->vmwarewinxp] NULL 512 1 1024 100 NULL 5 0 0 2007-04-04 09:45:38 1 0 vcl
> > 3244 x86 NULL NULL 0
> > 8 rh4image-VMwareserverhostRHEL48-v3 VMware server host (RHEL4) 1 [->admin]
> > 1 [->i386] 13 [->rh4image] NULL 64 1 500 10 NULL 10 0 0 2007-11-26 11:23:53
> > 1 0 vcl 1450 x86 NULL NULL 0
> > 9 esx35-base-v0 VMware ESX 3.5 standard server 1 [->admin] 1 [->i386] 20
> > [->esx35] NULL 2048 2 2000 100 NULL 9 0 0 2008-03-24 14:23:54 1 0 vcl 1450
> > x86 NULL NULL 0
> > 10 esx3-basewinxp-v0 No Apps (ESXi WinXP) 1 [->admin] 1 [->i386] 20
> > [->esx35] NULL 512 1 1024 100 NULL 9 0 0 2010-03-24 17:35:12 1 0 vcl 4100
> > x86 NULL NULL 0
> > /End Image Table====================
> >
> >
> > ImageRevision Table: Rows: 11
> > ===========================
> > id imageid revision userid datecreated deleted datedeleted
> > production comments imagename
> > 1 1 [->No Apps (WinXP)] 0 1 [->admin] 2007-03-01 14:46:26 0 NULL 1 NULL
> > winxp-base1-v0
> > 2 2 [->Red Hat Enterprise Linux 4.4 Base (KS)] 0 1 [->admin] 2006-09-22
> > 16:33:24 0 NULL 1 NULL rhel4-base2-v0
> > 3 3 [->RHEL4 base (image)] 0 1 [->admin] 2007-01-24 15:04:11 0 NULL 1 NULL
> > rh4image-base3-v0
> > 4 4 [->No Image] 0 1 [->admin] 1980-01-01 00:00:00 0 NULL 1 NULL noimage
> > 5 5 [->Red Hat Fedora Core 5 base (KS)] 0 1 [->admin] 2006-10-02 10:04:24 0
> > NULL 1 NULL rhfc5-fc5base5-v0
> > 6 6 [->FC5base(RHFC5)] 0 1 [->admin] 2006-10-02 14:13:26 0 NULL 1 NULL
> > fc5image-FC5baseRHFC56-v0
> > 7 7 [->No Apps (WinXP vmware)] 0 1 [->admin] 2007-04-03 11:24:12 0 NULL 1
> > NULL vmwarewinxp-base7-v0
> > 8 8 [->VMware server host (RHEL4)] 0 1 [->admin] 2007-01-30 10:50:56 0 NULL
> > 1 NULL rh4image-VMwareserverhostRHEL48-v0
> > 9 9 [->VMware ESX 3.5 standard server] 0 1 [->admin] 2008-03-24 14:23:54 0
> > NULL 1 NULL esx35-base-v0
> > 10 7 [->No Apps (WinXP vmware)] 1 1 [->admin] 2010-03-18 14:56:50 0 NULL 1
> > NULL vmwarewinxp-base7-v1
> > 11 10 [->No Apps (ESXi WinXP)] 0 1 [->admin] 2010-03-24 17:35:12 0 NULL 1
> > NULL esx3-basewinxp-v0
> > /End ImageRevision Table====================
> >
> >
> > Computer Table: Rows: 3
> > ============================================
> > id stateid ownerid platformid scheduleid currentimageid
> > preferredimageid nextimageid imagerevisionid RAM procnumber procspeed
> > network hostname IPaddress privateIPaddress eth0macaddress
> > eth1macaddress type provisioningid drivetype deleted notes lastcheck
> > location dsa dsapub rsa rsapub host hostpub vmhostid vmtypeid
> > 1 10 [->maintenance] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0
> > 0 0 512 2 2200 1000 resxi-2.odu.edu 128.82.128.218 NULL NULL NULL
> > virtualmachine 4 [->vmware_esx] hda 1 NULL NULL NULL NULL NULL NULL
> NULL
> > [BLOB - 0B] NULL NULL NULL
> > 2 2 [->available] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0
> > 0 20480 8 2200 1000 esx35-hypervisor-1.odu.edu 128.82.128.252 NULL NULL
> > NULL blade 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL [BLOB
> -
> > 0B] NULL NULL NULL
> > 3 5 [->failed] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0 0
> > 512 2 2200 100 esx3-basewinxp-v0.odu.edu 128.82.128.217 NULL NULL NULL
> > virtualmachine 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL
> [BLOB
> > - 0B] NULL 1 NULL
> > /End Computer Table======================================
> >
> >
> > Vmprofile Table: Rows: 4
> > ========================
> > id profilename vmtypeid imageid nasshare datastorepath vmpath
> > virtualswitch0 virtualswitch1 vmdisk username password
> > 1 VMware GSX standard 3 [->vmwareGSX] 8 [->VMware server host (RHEL4)] NULL
> > /var/lib/vmware/Virtual Machines NULL VMnet0 VMnet2 localdisk NULL NULL
> > 2 Vmware ESX standard network mounted share 5 [->vmwareESX3] 9 [->VMware
> > ESX 3.5 standard server] NULL /vmfs/volumes/nfs1 /vmfs/volumes/storage1 VM
> > Network Virtual Machine Public Network networkdisk NULL NULL
> > 3 Vmware ESX standard localdisk 6 [->vmwareESXi] 9 [->VMware ESX 3.5
> > standard server] NULL 192.168.129.37:/install/image/vmware_images
> > 192.168.129.37:/install/image/vmware_images VM Network Private Vlan328
> > localdisk root
> > 4 Vmware ESX SAN 5 [->vmwareESX3] 9 [->VMware ESX 3.5 standard server]
> NULL
> > /vmfs/volumes/NetApp /vmfs/volumes/storage1 Intranet2 MCNC Public
> > networkdisk NULL NULL
> > /End Vmprofile===============================
> >
> > VMhost Table: Rows: 1
> > =====================
> > id computerid vmlimit vmprofileid vmkernalnic vmwaredisk
> > 1 2 [->esx35-hypervisor-1.odu.edu] 2 3 NULL localdisk
> > /End Vmhost Table==========================
> >
> >
> > ================================= LOG FILE
> > ===========================================
> > 13 2010-04-01 15:37:19|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:19
> > 14 2010-04-01 15:37:24|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:24
> > 15 2010-04-01
> 15:37:24|5754|9:9|deleted|utils.pm:reservation_being_processed(9634)|computerloadlog
> > 'begin' entry does NOT exist for reservation 9
> > 16 2010-04-01
> 15:37:24|5754|9:9|deleted|utils.pm:reservation_being_processed(9686)|reservation
> > is NOT currently being processed
> > 17 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(276)|reservation 9 is
> > NOT already being processed
> > 18
> > 19 |5754|9:9|deleted| ---- WARNING ----
> > 20 |5754|9:9|deleted| 2010-04-01
> 15:37:24|5754|9:9|deleted|utils.pm:get_request_info(5218)|preferredimageid
> > is not set for computer id=2
> > 21 |5754|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 22 |5754|9:9|deleted| (-1) utils.pm, get_request_info (line: 5218)
> > 23 |5754|9:9|deleted| (-2) vcld, main (line: 280)
> > 24
> > 25 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:get_request_info(5354)|standalone
> > affiliation found: Local
> > 26 2010-04-01
> 15:37:24|5754|9:9|deleted|utils.pm:get_management_node_info(6815)|management
> > node info retrieved from database for colbert
> > 27 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(281)|retrieved
> > request information from database
> > 28 2010-04-01
> > 15:37:24|5754|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 9
> > 29 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:update_request_state(2186)|request
> > 9 state updated to: pending, laststate to: deleted
> > 30 2010-04-01
> 15:37:24|5754|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 31 2010-04-01
> > 15:37:24|5754|9:9|deleted|vcld:make_new_child(489)|request will be processed
> > by reclaim.pm
> > 32 2010-04-01 15:37:24|5754|9:9|deleted|vcld:make_new_child(509)|loaded
> > VCL::reclaim module
> > 33 2010-04-01 15:37:24|12719|9:9|deleted|vcld:make_new_child(582)|vcld
> > environment variable set to 0 for this process
> > 34 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> > called, class=VCL::reclaim
> > 35 2010-04-01
> > 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::reclaim object created
> > 36 2010-04-01
> > 15:37:24|5754|9:9|deleted|vcld:make_new_child(566)|current number of forked
> > kids: 1
> > 37 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(85)|obtained a database
> > handle for this state process, stored as $ENV{dbh}
> > 38 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:check_image_os(820)|no corrections need
> > to be made, not an imaging request, returning 1
> > 39 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 9
> > 40 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7863)|reservation
> > count: 1
> > 41 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7882)|PARENTIMAGE:
> > 1
> > 42 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7883)|SUBIMAGE:
> > 0
> > 43 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7901)|renamed
> > process to 'vcld VCL::reclaim 9:9 deleted'
> > 44 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 9
> > 45 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 9
> > 46 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(121)|attempting to load
> > provisioning module: VCL::Module::Provisioning::esx
> > 47 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(138)|esx
> > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > 48 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(140)|vmware
> > ESX module initialized
> > 49 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(127)|VCL::Module::Provisioning::esx
> > module loaded
> > 50 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> > called, class=VCL::Module::Provisioning::esx
> > 51 2010-04-01
> > 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::Provisioning::esx
> > object created
> > 52 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(138)|esx
> > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > 53 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(140)|vmware
> > ESX module initialized
> > 54 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(131)|VCL::Module::Provisioning::esx
> > provisioner object created
> > 55 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(145)|attempting to load OS
> > module: VCL::Module::OS::Linux
> > 56 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(151)|VCL::Module::OS::Linux
> > module loaded
> > 57 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> > called, class=VCL::Module::OS::Linux
> > 58 2010-04-01
> > 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::OS::Linux object
> > created
> > 59 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(154)|VCL::Module::OS::Linux
> > OS object created
> > 60 2010-04-01
> > 15:37:24|12719|9:9|deleted|State.pm:initialize(169)|returning 1
> > 61 2010-04-01
> > 15:37:24|12719|9:9|deleted|vcld:make_new_child(591)|VCL::reclaim object
> > created and initialized
> > 62 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation
> > remote IP is not defined
> > 63 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1946)|attempting
> > to retrieve current state of computer esx35-hypervisor-1 from the database
> > 64 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > current state of computer esx35-hypervisor-1 from the database: available
> > 65 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:_automethod(697)|data structure
> > updated: $self->request_data->{reservation}{9}{computer}{state}{name}
> > 66 |12719|9:9|deleted| computer_state_name = available
> > 67 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 68 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 69 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 70 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 71 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 72 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(138)|computer
> > type is blade
> > 73 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(142)|request
> > laststate is reserved, attempting to sanitize computer
> > 74 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(172)|calling
> > VCL::Module::OS::Linux::sanitize() subroutine
> > 75 2010-04-01
> > 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation
> > remote IP is not defined
> > 76 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 77 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'netstat -an' 2>&1
> > 78
> > 79 |12719|9:9|deleted| ---- WARNING ----
> > 80 |12719|9:9|deleted| 2010-04-01
> 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
> > status: 255, SSH exits with the exit status of the remote command or with
> > 255 if an error occurred, output:
> > 81 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> > found
> > 82 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 83 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 84 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > 85 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > 86 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 87 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 88 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 89
> > 90 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 91 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 92 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'netstat -an' 2>&1
> > 93
> > 94 |12719|9:9|deleted| ---- WARNING ----
> > 95 |12719|9:9|deleted| 2010-04-01
> 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
> > status: 255, SSH exits with the exit status of the remote command or with
> > 255 if an error occurred, output:
> > 96 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> > found
> > 97 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 98 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 99 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > 100 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > 101 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 102 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 103 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 104
> > 105 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 106 2010-04-01 15:37:29|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:29
> > 107 2010-04-01 15:37:34|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:34
> > 108 2010-04-01 15:37:39|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:39
> > 109 2010-04-01
> 15:37:39|5754|10:10|new|utils.pm:reservation_being_processed(9634)|computerloadlog
> > 'begin' entry does NOT exist for reservation 10
> > 110 2010-04-01
> 15:37:39|5754|10:10|new|utils.pm:reservation_being_processed(9686)|reservation
> > is NOT currently being processed
> > 111 2010-04-01 15:37:39|5754|10:10|new|vcld:main(276)|reservation 10 is
> > NOT already being processed
> > 112
> > 113 |5754|10:10|new| ---- WARNING ----
> > 114 |5754|10:10|new| 2010-04-01
> 15:37:39|5754|10:10|new|utils.pm:get_request_info(5218)|preferredimageid
> > is not set for computer id=3
> > 115 |5754|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 116 |5754|10:10|new| (-1) utils.pm, get_request_info (line: 5218)
> > 117 |5754|10:10|new| (-2) vcld, main (line: 280)
> > 118
> > 119 2010-04-01 15:37:39|5754|10:10|new|utils.pm:get_request_info(5354)|standalone
> > affiliation found: Local
> > 120 2010-04-01
> 15:37:39|5754|10:10|new|utils.pm:get_management_node_info(6815)|management
> > node info retrieved from database for colbert
> > 121 2010-04-01 15:37:39|5754|10:10|new|vcld:main(281)|retrieved request
> > information from database
> > 122 2010-04-01
> > 15:37:39|5754|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 123 2010-04-01 15:37:39|5754|10:10|new|utils.pm:update_request_state(2186)|request
> > 10 state updated to: pending, laststate to: new
> > 124 2010-04-01 15:37:39|5754|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, begin, beginning to process, state is new
> > 125 2010-04-01 15:37:39|5754|10:10|new|vcld:make_new_child(509)|loaded
> > VCL::new module
> > 126 2010-04-01 15:37:39|12726|10:10|new|vcld:make_new_child(582)|vcld
> > environment variable set to 0 for this process
> > 127 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> > called, class=VCL::new
> > 128 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::new
> > object created
> > 129 2010-04-01 15:37:39|5754|10:10|new|vcld:make_new_child(566)|current
> > number of forked kids: 2
> > 130 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(85)|obtained
> > a database handle for this state process, stored as $ENV{dbh}
> > 131 2010-04-01 15:37:39|12726|10:10|new|State.pm:check_image_os(820)|no
> > corrections need to be made, not an imaging request, returning 1
> > 132 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 133 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7863)|reservation
> > count: 1
> > 134 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7882)|PARENTIMAGE:
> > 1
> > 135 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7883)|SUBIMAGE:
> > 0
> > 136 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7901)|renamed
> > process to 'vcld VCL::new 10:10 new'
> > 137 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 138 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 139 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(121)|attempting to load
> > provisioning module: VCL::Module::Provisioning::esx
> > 140 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx
> > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > 141 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(140)|vmware
> > ESX module initialized
> > 142 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(127)|VCL::Module::Provisioning::esx
> > module loaded
> > 143 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> > called, class=VCL::Module::Provisioning::esx
> > 144 2010-04-01
> > 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::Provisioning::esx
> > object created
> > 145 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx
> > vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> > 146 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(140)|vmware
> > ESX module initialized
> > 147 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(131)|VCL::Module::Provisioning::esx
> > provisioner object created
> > 148 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(145)|attempting to load OS
> > module: VCL::Module::OS::Linux
> > 149 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(151)|VCL::Module::OS::Linux
> > module loaded
> > 150 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> > called, class=VCL::Module::OS::Linux
> > 151 2010-04-01
> > 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::OS::Linux object
> > created
> > 152 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(154)|VCL::Module::OS::Linux OS
> > object created
> > 153 2010-04-01
> > 15:37:39|12726|10:10|new|State.pm:initialize(169)|returning 1
> > 154 2010-04-01
> > 15:37:39|12726|10:10|new|vcld:make_new_child(591)|VCL::new object created
> > and initialized
> > 155 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 156 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > to retrieve current state of computer esx3-basewinxp-v0 from the database
> > 157 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > current state of computer esx3-basewinxp-v0 from the database: available
> > 158 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > 159 |12726|10:10|new| computer_state_name = available
> > 160
> > 161 |12726|10:10|new| ---- WARNING ----
> > 162 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_id:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> > 163 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 164 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 165 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 166 |12726|10:10|new| (-3) new.pm, process (line: 106)
> > 167 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> > 168 |12726|10:10|new| (-5) vcld, main (line: 341)
> > 169
> > 170
> > 171 |12726|10:10|new| ---- WARNING ----
> > 172 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_name:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> > 173 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 174 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 175 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 176 |12726|10:10|new| (-3) new.pm, process (line: 107)
> > 177 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> > 178 |12726|10:10|new| (-5) vcld, main (line: 341)
> > 179
> > 180 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
> > variable: 1
> > 181 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(134)|reservation
> > is parent = 1
> > 182 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(135)|preload
> > only = 0
> > 183 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(136)|originating
> > request state = new
> > 184 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(137)|originating
> > request laststate = new
> > 185 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(138)|originating
> > computer state = available
> > 186 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(139)|originating
> > computer type = virtualmachine
> > 187 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 188 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > to retrieve current state of computer esx3-basewinxp-v0 from the database
> > 189 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > current state of computer esx3-basewinxp-v0 from the database: available
> > 190 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > 191 |12726|10:10|new| computer_state_name = available
> > 192
> > 193 |12726|10:10|new| ---- WARNING ----
> > 194 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_id:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> > 195 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 196 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 197 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 198 |12726|10:10|new| (-3) new.pm, computer_not_being_used (line: 750)
> > 199 |12726|10:10|new| (-4) new.pm, process (line: 170)
> > 200 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 201 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 202
> > 203
> > 204 |12726|10:10|new| ---- WARNING ----
> > 205 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_name:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> > 206 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 207 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 208 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 209 |12726|10:10|new| (-3) new.pm, computer_not_being_used (line: 751)
> > 210 |12726|10:10|new| (-4) new.pm, process (line: 170)
> > 211 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 212 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 213
> > 214 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
> > variable: 1
> > 215 2010-04-01
> 15:37:39|12726|10:10|new|new.pm:computer_not_being_used(788)|esx3-basewinxp-v0
> > state is available
> > 216 2010-04-01
> 15:37:39|12726|10:10|new|new.pm:computer_not_being_used(798)|esx3-basewinxp-v0
> > is available, its state is available
> > 217 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(170)|esx3-basewinxp-v0
> > is not being used
> > 218 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> > true: parent reservation ID for this request: 10
> > 219 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > to retrieve current state of computer esx3-basewinxp-v0 from the database
> > 220 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > current state of computer esx3-basewinxp-v0 from the database: available
> > 221 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > 222 |12726|10:10|new| computer_state_name = available
> > 223
> > 224 |12726|10:10|new| ---- WARNING ----
> > 225 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_id:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> > 226 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 227 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 228 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 229 |12726|10:10|new| (-3) new.pm, reload_image (line: 480)
> > 230 |12726|10:10|new| (-4) new.pm, process (line: 266)
> > 231 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 232 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 233
> > 234
> > 235 |12726|10:10|new| ---- WARNING ----
> > 236 |12726|10:10|new| 2010-04-01
> > 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_preferredimage_name:
> > $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> > 237 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 238 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 239 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 240 |12726|10:10|new| (-3) new.pm, reload_image (line: 481)
> > 241 |12726|10:10|new| (-4) new.pm, process (line: 266)
> > 242 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 243 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 244
> > 245 2010-04-01
> 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
> > variable: 1
> > 246 2010-04-01 15:37:39|12726|10:10|new|new.pm:reload_image(514)|calling
> > VCL::Module::Provisioning::esx->node_status()
> > 247 2010-04-01 15:37:39|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, statuscheck, checking status of node
> > 248 $VAR1 = bless( {
> > 249 'data' => bless( do{\(my $o = 1)},
> > 'VCL::DataStructure' ),
> > 250 'os' => bless( {
> > 251 'provisioner' => $VAR1,
> > 252 'data' => $VAR1->{'data'}
> > 253 }, 'VCL::Module::OS::Linux' )
> > 254 }, 'VCL::Module::Provisioning::esx' );
> > 255 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(787)|Entering
> > node_status, checking status of esx3-basewinxp-v0
> > 256 2010-04-01
> 15:37:39|12726|10:10|new|esx.pm:node_status(788)|request_for_imaging:
> > 0
> > 257 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(789)|requeseted
> > image name: esx3-basewinxp-v0
> > 258 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(819)|checking
> > if esx3-basewinxp-v0 is pingable
> > 259 2010-04-01 15:37:40|12726|10:10|new|esx.pm:node_status(825)|esx3-basewinxp-
> v0
> > is not pingable (0)
> > 260 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(560)|node_status
> > returned a scalar: RELOAD
> > 261 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(602)|node
> > status is RELOAD, esx3-basewinxp-v0 will be reloaded
> > 262 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, loadimageblade, esx3-basewinxp-v0 must be reloaded with
> > esx3-basewinxp-v0
> > 263 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(616)|calling
> > VCL::Module::Provisioning::esx->does_image_exist()
> > 264 2010-04-01
> 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37:
> > 265 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'ls -1 /install/image/vmware_images/golden 2>&1' 2>&1
> > 266 2010-04-01
> 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 267 |12726|10:10|new| esx3-basewinxp-v0
> > 268 2010-04-01 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "esx3-basewinxp-v0")
> > 269 2010-04-01 15:37:40|12726|10:10|new|esx.pm:does_image_exist(917)|image
> > esx3-basewinxp-v0 exists
> > 270 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(619)|esx3-
> basewinxp-v0
> > exists on this management node
> > 271 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, doesimageexists, confirmed image exists
> > 272 2010-04-01
> 15:37:40|12726|10:10|new|utils.pm:update_computer_state(2228)|computer
> > 3 state updated to: reloading
> > 273 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(652)|computer
> > esx3-basewinxp-v0 state set to reloading
> > 274 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, info, computer state updated to reloading
> > 275 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(663)|calling
> > VCL::Module::Provisioning::esx->load() subroutine
> > 276 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, info, calling VCL::Module::Provisioning::esx->load() subroutine
> > 277 2010-04-01 15:37:40|12726|10:10|new|esx.pm:
> > load(165)|****************************************************
> > 278
> > 279 |12726|10:10|new| ---- WARNING ----
> > 280 |12726|10:10|new| 2010-04-01
> > 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_eth0_mac_address:
> > $self->request_data->{reservation}{10}{computer}{eth0macaddress}
> > 281 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 282 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 283 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 284 |12726|10:10|new| (-3) esx.pm, load (line: 181)
> > 285 |12726|10:10|new| (-4) new.pm, reload_image (line: 665)
> > 286 |12726|10:10|new| (-5) new.pm, process (line: 266)
> > 287 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
> > 288
> > 289
> > 290 |12726|10:10|new| ---- WARNING ----
> > 291 |12726|10:10|new| 2010-04-01
> > 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> > data has not been initialized for get_computer_eth1_mac_address:
> > $self->request_data->{reservation}{10}{computer}{eth1macaddress}
> > 292 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 293 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> > 294 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> > 295 |12726|10:10|new| (-3) esx.pm, load (line: 182)
> > 296 |12726|10:10|new| (-4) new.pm, reload_image (line: 665)
> > 297 |12726|10:10|new| (-5) new.pm, process (line: 266)
> > 298 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
> > 299
> > 300 2010-04-01 15:37:40|12726|10:10|new|esx.pm:load(193)|Calling
> > get_vmware_host_info
> > 301 No recipient addresses found in header
> > 302 2010-04-01 15:37:41|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
> > Sending mail To: , PROBLEM -- esx.pm
> > 303
> > 304 |12726|10:10|new| ---- CRITICAL ----
> > 305 |12726|10:10|new| 2010-04-01
> 15:37:40|12726|10:10|new|esx.pm:get_vmware_host_info(1012)|AMIT
> > DEBUGGING: esx35-hypervisor-1.odu.edu
> > 306 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 307 |12726|10:10|new| (-1) esx.pm, get_vmware_host_info (line: 1012)
> > 308 |12726|10:10|new| (-2) esx.pm, load (line: 194)
> > 309 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
> > 310 |12726|10:10|new| (-4) new.pm, process (line: 266)
> > 311 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 312 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 313
> > 314 2010-04-01 15:37:41|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 315 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'netstat -an' 2>&1
> > 316
> > 317 |12719|9:9|deleted| ---- WARNING ----
> > 318 |12719|9:9|deleted| 2010-04-01
> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 3/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
> > status: 255, SSH exits with the exit status of the remote command or with
> > 255 if an error occurred, output:
> > 319 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> > found
> > 320 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 321 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 322 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > 323 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > 324 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 325 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 326 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 327
> > 328
> > 329 |12719|9:9|deleted| ---- WARNING ----
> > 330 |12719|9:9|deleted| 2010-04-01
> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'netstat -an' 2>&1,
> > exit status: 255, output:
> > 331 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> > found
> > 332 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 333 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
> > 334 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> > 335 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> > 336 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 337 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 338 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 339
> > 340 2010-04-01
> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 341 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > /etc/ssh/external_sshd_config"' 2>&1
> > 342
> > 343 |12719|9:9|deleted| ---- WARNING ----
> > 344 |12719|9:9|deleted| 2010-04-01
> 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx
> > "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255, SSH
> > exits with the exit status of the remote command or with 255 if an error
> > occurred, output:
> > 345 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not
> > found
> > 346 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 347 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 348 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line: 913)
> > 349 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
> > 350 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
> > 351 |12719|9:9|deleted| (-5) reclaim.pm, process (line: 173)
> > 352 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
> > 353
> > 354 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 355 2010-04-01 15:37:43|12726|10:10|new|esx.pm:get_vmware_host_info(1019)|host
> > info output for esx35-hypervisor-1
> > 356 |12726|10:10|new| Host Information
> > 357 |12726|10:10|new| Host Name: esx35-hypervisor-1.odu.edu
> > 358 2010-04-01 15:37:43|12726|10:10|new|esx.pm:get_vmware_host_info(1025)|found
> > hostname_value= esx35-hypervisor-1.odu.edu
> > 359 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(197)|Collected
> > esx35-hypervisor-1.odu.edu for vmware host name
> > 360 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(210)|DATASTORE IP
> > is 192.168.129.37 and DATASTORE_SHARE_PATH is /install/image/vmware_images
> > 361 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(211)|Entered ESX
> > module, loading esx3-basewinxp-v0 on esx3-basewinxp-v0 (on
> > esx35-hypervisor-1.odu.edu) for reservation 10
> > 362 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(212)|Datastore:
> > 192.168.129.37:/install/image/vmware_images
> > 363 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(223)|VM info
> > command: /usr/lib/vmware-vcli/apps/vm/vminfo.pl --server
> > 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --username root --password
> > 'xxxxxxxxxxx'
> > 364 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 365 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > /etc/ssh/external_sshd_config"' 2>&1
> > 366 2010-04-01 15:37:44|12726|10:10|new|esx.pm:load(226)|VM info
> > output: Virtual Machine esx3-basewinxp-v0 not found.
> > 367 2010-04-01
> 15:37:44|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37:
> > 368 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'rm -rf /install/image/vmware_images/inuse/esx3-basewinxp-v0'
> > 2>&1
> > 369 2010-04-01 15:37:44|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:44
> > 370
> > 371 |12719|9:9|deleted| ---- WARNING ----
> > 372 |12719|9:9|deleted| 2010-04-01
> 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx
> > "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255, SSH
> > exits with the exit status of the remote command or with 255 if an error
> > occurred, output:
> > 373 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not
> > found
> > 374 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 375 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 376 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line: 913)
> > 377 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
> > 378 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
> > 379 |12719|9:9|deleted| (-5) reclaim.pm, process (line: 173)
> > 380 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
> > 381
> > 382 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 383 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 384 |12726|10:10|new| none
> > 385 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "none")
> > 386 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(262)|Removed old vm
> > folder
> > 387 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37:
> > 388 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'mkdir /install/image/vmware_images/inuse/esx3-basewinxp-v0'
> > 2>&1
> > 389 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 390 |12726|10:10|new| none
> > 391 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "none")
> > 392 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37:
> > 393 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'cp
> > /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk
> > /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk'
> > 2>&1
> > 394 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 395 |12726|10:10|new| none
> > 396 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "none")
> > 397 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(278)|COPIED VMDK
> > SUCCESSFULLY
> > 398 2010-04-01
> 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37:
> > 399 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'cp
> > /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk
> > /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk'
> > 2>&1
> > 400 2010-04-01 15:37:49|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:49
> > 401 2010-04-01 15:37:54|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:54
> > 402 2010-04-01 15:37:59|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 403 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > /etc/ssh/external_sshd_config"' 2>&1
> > 404 2010-04-01 15:37:59|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:37:59
> > 405 2010-04-01
> 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 406 |12719|9:9|deleted| ash: pkill: not found
> > 407 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6272)|SSH
> > command executed on esx35-hypervisor-1, command:
> > 408 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> > /etc/ssh/external_sshd_config"' 2>&1
> > 409 |12719|9:9|deleted| returning (127, "ash: pkill: not found")
> > 410 2010-04-01
> > 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(916)|output detected:
> > ash: pkill: not found
> > 411 2010-04-01
> > 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(921)|ext_sshd on
> > esx35-hypervisor-1 stopped
> > 412 2010-04-01
> > 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(884)|stopped external sshd
> > 413 2010-04-01
> > 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(888)|access has been
> > revoked to esx35-hypervisor-1
> > 414 2010-04-01
> 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 415 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 416
> > 417 |12719|9:9|deleted| ---- WARNING ----
> > 418 |12719|9:9|deleted| 2010-04-01
> 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 419 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 420 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 421 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 422 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 423 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 424 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 425 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > 426 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 427 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 428 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 429 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 430
> > 431 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 432 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 433 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 434
> > 435 |12719|9:9|deleted| ---- WARNING ----
> > 436 |12719|9:9|deleted| 2010-04-01
> 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 437 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 438 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 439 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 440 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 441 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 442 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 443 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > 444 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 445 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 446 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 447 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 448
> > 449 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 450 2010-04-01 15:38:04|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:04
> > 451 2010-04-01 15:38:09|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:09
> > 452 2010-04-01 15:38:14|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:14
> > 453 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 454 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 455
> > 456 |12719|9:9|deleted| ---- WARNING ----
> > 457 |12719|9:9|deleted| 2010-04-01
> 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 3/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 458 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 459 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 460 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 461 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 462 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 463 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 464 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > 465 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 466 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 467 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 468 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 469
> > 470
> > 471 |12719|9:9|deleted| ---- WARNING ----
> > 472 |12719|9:9|deleted| 2010-04-01
> 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel
> > admin' 2>&1, exit status: 255, output:
> > 473 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 474 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 475 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 476 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 477 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 478 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
> > 479 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> > 480 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 481 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 482 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 483 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 484
> > 485 2010-04-01
> 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 486 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 487
> > 488 |12719|9:9|deleted| ---- WARNING ----
> > 489 |12719|9:9|deleted| 2010-04-01
> 15:38:18|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 490 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 491 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 492 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 493 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 494 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 495 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 496 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > 497 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 498 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 499 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 500 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 501
> > 502 2010-04-01 15:38:18|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 503 2010-04-01 15:38:19|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:19
> > 504 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 505 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 506
> > 507 |12719|9:9|deleted| ---- WARNING ----
> > 508 |12719|9:9|deleted| 2010-04-01
> 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 509 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 510 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 511 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 512 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 513 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 514 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 515 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > 516 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 517 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 518 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 519 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 520
> > 521 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 522 2010-04-01 15:38:25|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:24
> > 523 2010-04-01 15:38:30|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:30
> > 524 2010-04-01 15:38:35|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:35
> > 525 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 526 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> > 527
> > 528 |12719|9:9|deleted| ---- WARNING ----
> > 529 |12719|9:9|deleted| 2010-04-01
> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 3/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> > admin, exit status: 255, SSH exits with the exit status of the remote
> > command or with 255 if an error occurred, output:
> > 530 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 531 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 532 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 533 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 534 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 535 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 536 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > 537 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 538 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 539 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 540 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 541
> > 542
> > 543 |12719|9:9|deleted| ---- WARNING ----
> > 544 |12719|9:9|deleted| 2010-04-01
> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel
> > admin' 2>&1, exit status: 255, output:
> > 545 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/passwd'
> > 546 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/shadow'
> > 547 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/group'
> > 548 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> > 'admin' in '/etc/gshadow'
> > 549 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 550 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
> > 551 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> > 552 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 553 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 554 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 555 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 556
> > 557 2010-04-01
> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 558 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> > /etc/ssh/external_sshd_config' 2>&1
> > 559
> > 560 |12719|9:9|deleted| ---- WARNING ----
> > 561 |12719|9:9|deleted| 2010-04-01
> 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> > "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
> > exits with the exit status of the remote command or with 255 if an error
> > occurred, output:
> > 562 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > /etc/ssh/external_sshd_config: No such file or directory
> > 563 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 564 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 565 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > 566 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 567 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 568 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 569 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 570
> > 571 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 572 2010-04-01 15:38:37|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 573 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> > /etc/ssh/external_sshd_config' 2>&1
> > 574
> > 575 |12719|9:9|deleted| ---- WARNING ----
> > 576 |12719|9:9|deleted| 2010-04-01
> 15:38:38|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> > "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
> > exits with the exit status of the remote command or with 255 if an error
> > occurred, output:
> > 577 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > /etc/ssh/external_sshd_config: No such file or directory
> > 578 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 579 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 580 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > 581 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 582 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 583 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 584 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 585
> > 586 2010-04-01 15:38:38|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 587 2010-04-01 15:38:40|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:40
> > 588 2010-04-01 15:38:45|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:45
> > 589 2010-04-01 15:38:50|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:50
> > 590 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 591 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> > /etc/ssh/external_sshd_config' 2>&1
> > 592
> > 593 |12719|9:9|deleted| ---- WARNING ----
> > 594 |12719|9:9|deleted| 2010-04-01
> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> > "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
> > exits with the exit status of the remote command or with 255 if an error
> > occurred, output:
> > 595 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > /etc/ssh/external_sshd_config: No such file or directory
> > 596 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 597 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 598 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > 599 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 600 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 601 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 602 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 603
> > 604
> > 605 |12719|9:9|deleted| ---- WARNING ----
> > 606 |12719|9:9|deleted| 2010-04-01
> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^AllowUsers
> > .*/d" /etc/ssh/external_sshd_config' 2>&1, exit status: 255, output:
> > 607 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> > /etc/ssh/external_sshd_config: No such file or directory
> > 608 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 609 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
> > 610 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> > 611 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 612 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 613 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 614 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 615
> > 616 No recipient addresses found in header
> > 617 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:mail(1348)|SUCCESS
> > -- Sending mail To: , PROBLEM -- Linux.pm
> > 618
> > 619 |12719|9:9|deleted| ---- CRITICAL ----
> > 620 |12719|9:9|deleted| 2010-04-01
> > 15:38:53|12719|9:9|deleted|Linux.pm:delete_user(566)|failed to add
> > AllowUsers admin to external_sshd_config
> > 621 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 622 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 566)
> > 623 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
> > 624 |12719|9:9|deleted| (-3) reclaim.pm, process (line: 173)
> > 625 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
> > 626 |12719|9:9|deleted| (-5) vcld, main (line: 341)
> > 627
> > 628 2010-04-01
> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> > SSH command on esx35-hypervisor-1:
> > 629 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> > 630
> > 631 |12719|9:9|deleted| ---- WARNING ----
> > 632 |12719|9:9|deleted| 2010-04-01
> 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
> > .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
> > remote command or with 255 if an error occurred, output:
> > 633 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> > No such file or directory
> > 634 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 635 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 636 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > 637 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 638 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 639 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 640 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 641
> > 642 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 2 seconds before making next SSH attempt
> > 643 2010-04-01 15:38:55|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:38:55
> > 644 2010-04-01 15:38:55|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 2/3: executing SSH command on esx35-hypervisor-1:
> > 645 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> > 646
> > 647 |12719|9:9|deleted| ---- WARNING ----
> > 648 |12719|9:9|deleted| 2010-04-01
> 15:38:56|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
> > .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
> > remote command or with 255 if an error occurred, output:
> > 649 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> > No such file or directory
> > 650 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 651 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 652 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > 653 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 654 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 655 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 656 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 657
> > 658 2010-04-01 15:38:56|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> > for 15 seconds before making next SSH attempt
> > 659 2010-04-01 15:39:00|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:00
> > 660 2010-04-01 15:39:05|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:05
> > 661 2010-04-01 15:39:10|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:10
> > 662 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> > 3/3: executing SSH command on esx35-hypervisor-1:
> > 663 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> > -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> > 664
> > 665 |12719|9:9|deleted| ---- WARNING ----
> > 666 |12719|9:9|deleted| 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> > 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
> > .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
> > remote command or with 255 if an error occurred, output:
> > 667 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> > No such file or directory
> > 668 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 669 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> > 670 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > 671 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 672 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 673 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 674 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 675
> > 676
> > 677 |12719|9:9|deleted| ---- WARNING ----
> > 678 |12719|9:9|deleted| 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> > to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> > /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^admin
> > .*/d" /etc/sudoers' 2>&1, exit status: 255, output:
> > 679 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> > No such file or directory
> > 680 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 681 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
> > 682 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> > 683 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> > 684 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> > 685 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> > 686 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> > 687
> > 688 No recipient addresses found in header
> > 689 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:mail(1348)|SUCCESS
> > -- Sending mail To: , PROBLEM -- Linux.pm
> > 690
> > 691 |12719|9:9|deleted| ---- CRITICAL ----
> > 692 |12719|9:9|deleted| 2010-04-01
> > 15:39:11|12719|9:9|deleted|Linux.pm:delete_user(578)|failed to clear admin
> > from /etc/sudoers
> > 693 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> > 694 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 578)
> > 695 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
> > 696 |12719|9:9|deleted| (-3) reclaim.pm, process (line: 173)
> > 697 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
> > 698 |12719|9:9|deleted| (-5) vcld, main (line: 341)
> > 699
> > 700 2010-04-01 15:39:11|12719|9:9|deleted|Linux.pm:sanitize(852)|users
> > have been deleted from esx35-hypervisor-1
> > 701 2010-04-01 15:39:11|12719|9:9|deleted|reclaim.pm:process(173)|esx35-
> hypervisor-1
> > has been sanitized
> > 702 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8018)|called
> > from VCL::reclaim::process(271)
> > 703 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8054)|parent:
> > parent reservation ID for this request: 9
> > 704 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 705 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:update_request_state(2186)|request
> > 9 state updated to: complete, laststate to: deleted
> > 706 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8078)|req=9:
> > request state changed: deleted->complete, laststate: reserved->deleted
> > 707 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 708 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 709 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:update_computer_state(2228)|computer
> > 2 state updated to: available
> > 710 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8096)|req=9:
> > computer esx35-hypervisor-1 state changed: available->available
> > 711 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8106)|req=9:
> > log table id=10 will not be updated
> > 712 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> > entry not inserted, reservation has been deleted
> > 713 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8122)|req=9:
> > process exiting
> > 714 2010-04-01
> > 15:39:11|12719|9:9|deleted|State.pm:DESTROY(905)|destructor called,
> > ref($self)=VCL::reclaim
> > 715 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:delete_computerloadlog_reservation(7551)|removing
> > computerloadlog entries matching loadstate = begin
> > 716 2010-04-01
> 15:39:11|12719|9:9|deleted|utils.pm:delete_computerloadlog_reservation(7598)|deleted
> > rows from computerloadlog for reservation id=9
> > 717 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(912)|removed
> > computerloadlog rows with loadstate=begin for reservation
> > 718 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(924)|number
> > of database handles state process created: 1
> > 719 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(933)|process
> > has a database handle stored in $ENV{dbh}, attempting disconnect
> > 720 2010-04-01
> > 15:39:11|12719|9:9|deleted|State.pm:DESTROY(935)|$ENV{dbh}: database
> > disconnect successful
> > 721 2010-04-01
> > 15:39:11|12719|9:9|deleted|State.pm:DESTROY(949)|VCL::reclaim process 12719
> > exiting
> > 722 2010-04-01 15:39:11|5754|vcld:REAPER(744)|VCL process exited for
> > reservation 9
> > 723 2010-04-01 15:39:11|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:11
> > 724 2010-04-01 15:39:11|5754|9:9|complete|vcld:main(250)|request deleted
> > 725 2010-04-01 15:39:16|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:16
> > 726 2010-04-01 15:39:21|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:21
> > 727 2010-04-01 15:39:26|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:26
> > 728 2010-04-01 15:39:31|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:31
> > 729 2010-04-01 15:39:36|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:36
> > 730 2010-04-01 15:39:42|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:42
> > 731 2010-04-01 15:39:47|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:47
> > 732 2010-04-01 15:39:52|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:52
> > 733 2010-04-01
> 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 734 |12726|10:10|new| none
> > 735 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "none")
> > 736 2010-04-01
> 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> > SSH command on 192.168.129.37:
> > 737 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> > 192.168.129.37 'grep adapterType
> > /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk
> > 2>&1' 2>&1
> > 738 2010-04-01
> 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> > output:
> > 739 |12726|10:10|new| ddb.adapterType = "buslogic"
> > 740 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> > command executed on 192.168.129.37, returning (0, "ddb.adapterType =
> > "buslogic"")
> > 741 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(311)|adapter=
> > buslogic
> > 742 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(342)|eth0 MAC
> > address set for vcl assigned
> > 743 Use of uninitialized value in concatenation (.) or string at
> > 744 /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pmline 343 (#1)
> > 745 (W uninitialized) An undefined value was used as if it were
> > already
> > 746 defined. It was interpreted as a "" or a 0, but maybe it was a
> > mistake.
> > 747 To suppress this warning assign a defined value to your
> > variables.
> > 748
> > 749 To help you figure out what was undefined, perl tells you what
> > operation
> > 750 you used the undefined value in. Note, however, that perl
> > optimizes your
> > 751 program and the operation displayed in the warning may not
> > necessarily
> > 752 appear literally in your program. For example, "that $foo" is
> > 753 usually optimized into "that " . $foo, and the warning will
> > refer to
> > 754 the concatenation (.) operator, even though there is no . in
> > your
> > 755 program.
> > 756
> > 757
> > 758 |12726|10:10|new| ---- WARNING ----
> > 759 |12726|10:10|new| 2010-04-01
> > 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> > value in concatenation (.) or string at
> > /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 343.
> > 760 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 761 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> > 762 |12726|10:10|new| (-2) esx.pm, load (line: 343)
> > 763 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
> > 764 |12726|10:10|new| (-4) new.pm, process (line: 266)
> > 765 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 766 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 767
> > 768
> > 769 |12726|10:10|new| ---- WARNING ----
> > 770 |12726|10:10|new| 2010-04-01
> > 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> > value in concatenation (.) or string at
> > /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 353.
> > 771 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 772 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> > 773 |12726|10:10|new| (-2) esx.pm, load (line: 353)
> > 774 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
> > 775 |12726|10:10|new| (-4) new.pm, process (line: 266)
> > 776 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 777 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 778
> > 779 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(353)|eth1 MAC
> > address set for vcl assigned
> > 780
> > 781 |12726|10:10|new| ---- WARNING ----
> > 782 |12726|10:10|new| 2010-04-01
> > 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> > value in concatenation (.) or string at
> > /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 354.
> > 783 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 784 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> > 785 |12726|10:10|new| (-2) esx.pm, load (line: 354)
> > 786 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
> > 787 |12726|10:10|new| (-4) new.pm, process (line: 266)
> > 788 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> > 789 |12726|10:10|new| (-6) vcld, main (line: 341)
> > 790
> > 791 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(376)|wrote vmxarray
> > to /tmp/esx3-basewinxp-v0.vmx
> > 792 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_scp_command(6403)|attempt
> > 1/3: executing SCP command: /usr/bin/scp -B -i /etc/vcl/vcl.key -P 22 -p -r
> > /tmp/esx3-basewinxp-v0.vmx 192.168.129.37:/install/image/vmware_images/inuse/esx3-
> basewinxp-v0/esx3-basewinxp-v0.vmx
> > 2>&1
> > 793 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_scp_command(6453)|scp
> > successful: attempt 1/3, exit status: 0, output: none
> > 794 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(401)|Register
> > Command: /usr/lib/vmware-vcli/apps/vm/vmregister.pl --server
> > 'esx35-hypervisor-1' --username root --password 'xxxxxxxxxxx' --vmxpath
> > '[VCL]/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmx' --operation register
> > --vmname esx3-basewinxp-v0 --pool Resources --hostname '
> > esx35-hypervisor-1.odu.edu' --datacenter 'ha-datacenter'
> > 795 2010-04-01 15:39:57|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:39:57
> > 796 2010-04-01 15:39:57|12726|10:10|new|esx.pm:load(404)|Registered:
> > 797 |12726|10:10|new| SOAP Fault:
> > 798 |12726|10:10|new| -----------
> > 799 |12726|10:10|new| Fault string: fault.RestrictedVersion.summary
> > 800 |12726|10:10|new| Fault detail: RestrictedVersionFault
> > 801 2010-04-01 15:39:57|12726|10:10|new|esx.pm:load(413)|Power on
> > command: /usr/lib/vmware-vcli/apps/vm/vmcontrol.pl --server
> > 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --operation poweron
> > --username root --password 'xxxxxxxxxxx'
> > 802 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(416)|Powered on:
> > Virtual Machine esx3-basewinxp-v0 not found.
> > 803 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(505)|IP is known
> > for esx3-basewinxp-v0
> > 804 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(511)|Waiting for
> > ssh to come up on esx3-basewinxp-v0
> > 805 2010-04-01 15:40:00|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 1
> > 806 2010-04-01 15:40:02|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:02
> > 807 2010-04-01 15:40:06|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 2
> > 808 2010-04-01 15:40:07|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:07
> > 809 2010-04-01 15:40:12|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:12
> > 810 2010-04-01 15:40:12|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 3
> > 811 2010-04-01 15:40:17|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:17
> > 812 2010-04-01 15:40:18|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 4
> > 813 2010-04-01 15:40:22|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:22
> > 814 2010-04-01 15:40:24|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 5
> > 815 2010-04-01 15:40:27|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:27
> > 816 2010-04-01 15:40:30|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 6
> > 817 2010-04-01 15:40:32|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:32
> > 818 2010-04-01 15:40:36|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 7
> > 819 2010-04-01 15:40:37|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:37
> > 820 2010-04-01 15:40:42|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:42
> > 821 2010-04-01 15:40:42|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 8
> > 822 2010-04-01 15:40:47|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:47
> > 823 2010-04-01 15:40:48|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 9
> > 824 2010-04-01 15:40:52|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:52
> > 825 2010-04-01 15:40:54|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 10
> > 826 2010-04-01 15:40:57|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:40:57
> > 827 2010-04-01 15:41:00|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 11
> > 828 2010-04-01 15:41:02|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:02
> > 829 2010-04-01 15:41:06|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 12
> > 830 2010-04-01 15:41:07|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:07
> > 831 2010-04-01 15:41:12|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:12
> > 832 2010-04-01 15:41:12|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 13
> > 833 2010-04-01 15:41:17|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:17
> > 834 2010-04-01 15:41:18|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 14
> > 835 2010-04-01 15:41:22|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:22
> > 836 2010-04-01 15:41:24|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 15
> > 837 2010-04-01 15:41:27|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:27
> > 838 2010-04-01 15:41:30|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 16
> > 839 2010-04-01 15:41:32|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:32
> > 840 2010-04-01 15:41:36|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 17
> > 841 2010-04-01 15:41:37|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:37
> > 842 2010-04-01 15:41:42|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:42
> > 843 2010-04-01 15:41:42|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 18
> > 844 2010-04-01 15:41:47|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:47
> > 845 2010-04-01 15:41:48|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 19
> > 846 2010-04-01 15:41:52|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:52
> > 847 2010-04-01 15:41:54|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 20
> > 848 2010-04-01 15:41:57|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:41:57
> > 849 2010-04-01 15:42:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 21
> > 850 2010-04-01 15:42:02|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:02
> > 851 2010-04-01 15:42:07|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 22
> > 852 2010-04-01 15:42:07|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:07
> > 853 2010-04-01 15:42:12|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:12
> > 854 2010-04-01 15:42:13|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 23
> > 855 2010-04-01 15:42:18|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:18
> > 856 2010-04-01 15:42:19|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 24
> > 857 2010-04-01 15:42:23|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:23
> > 858 2010-04-01 15:42:25|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 25
> > 859 2010-04-01 15:42:28|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:28
> > 860 2010-04-01 15:42:31|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 26
> > 861 2010-04-01 15:42:33|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:33
> > 862 2010-04-01 15:42:37|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 27
> > 863 2010-04-01 15:42:38|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:38
> > 864 2010-04-01 15:42:43|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 28
> > 865 2010-04-01 15:42:43|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:43
> > 866 2010-04-01 15:42:48|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:48
> > 867 2010-04-01 15:42:49|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 29
> > 868 2010-04-01 15:42:53|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:53
> > 869 2010-04-01 15:42:55|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 30
> > 870 2010-04-01 15:42:58|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:42:58
> > 871 2010-04-01 15:43:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 31
> > 872 2010-04-01 15:43:03|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:03
> > 873 2010-04-01 15:43:07|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 32
> > 874 2010-04-01 15:43:08|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:08
> > 875 2010-04-01 15:43:13|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 33
> > 876 2010-04-01 15:43:13|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:13
> > 877 2010-04-01 15:43:18|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:18
> > 878 2010-04-01 15:43:19|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 34
> > 879 2010-04-01 15:43:23|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:23
> > 880 2010-04-01 15:43:25|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 35
> > 881 2010-04-01 15:43:28|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:28
> > 882 2010-04-01 15:43:31|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 36
> > 883 2010-04-01 15:43:33|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:33
> > 884 2010-04-01 15:43:37|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 37
> > 885 2010-04-01 15:43:38|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:38
> > 886 2010-04-01 15:43:43|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 38
> > 887 2010-04-01 15:43:43|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:43
> > 888 2010-04-01 15:43:48|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:48
> > 889 2010-04-01 15:43:49|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 39
> > 890 2010-04-01 15:43:53|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:53
> > 891 2010-04-01 15:43:55|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 40
> > 892 2010-04-01 15:43:58|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:43:58
> > 893 2010-04-01 15:44:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 41
> > 894 2010-04-01 15:44:03|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:03
> > 895 2010-04-01 15:44:07|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 42
> > 896 2010-04-01 15:44:08|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:08
> > 897 2010-04-01 15:44:13|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 43
> > 898 2010-04-01 15:44:13|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:13
> > 899 2010-04-01 15:44:18|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:18
> > 900 2010-04-01 15:44:19|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 44
> > 901 2010-04-01 15:44:23|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:23
> > 902 2010-04-01 15:44:25|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 45
> > 903 2010-04-01 15:44:29|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:28
> > 904 2010-04-01 15:44:31|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 46
> > 905 2010-04-01 15:44:34|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:34
> > 906 2010-04-01 15:44:37|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 47
> > 907 2010-04-01 15:44:39|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:39
> > 908 2010-04-01 15:44:43|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 48
> > 909 2010-04-01 15:44:44|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:44
> > 910 2010-04-01 15:44:49|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:49
> > 911 2010-04-01 15:44:49|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 49
> > 912 2010-04-01 15:44:54|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:54
> > 913 2010-04-01 15:44:55|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 50
> > 914 2010-04-01 15:44:59|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:44:59
> > 915 2010-04-01 15:45:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> > 5 seconds, waiting for computer to start SSH. Try 51
> > 916 2010-04-01 15:45:04|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:45:04
> > 917 No recipient addresses found in header
> > 918 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
> > Sending mail To: , PROBLEM -- esx.pm
> > 919
> > 920 |12726|10:10|new| ---- CRITICAL ----
> > 921 |12726|10:10|new| 2010-04-01 15:45:07|12726|10:10|new|esx.pm:load(521)|waited
> > acceptable amount of time for sshd to become active, please check
> > esx3-basewinxp-v0 on esx35-hypervisor-1
> > 922 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 923 |12726|10:10|new| (-1) esx.pm, load (line: 521)
> > 924 |12726|10:10|new| (-2) new.pm, reload_image (line: 665)
> > 925 |12726|10:10|new| (-3) new.pm, process (line: 266)
> > 926 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> > 927 |12726|10:10|new| (-5) vcld, main (line: 341)
> > 928
> > 929
> > 930 |12726|10:10|new| ---- WARNING ----
> > 931 |12726|10:10|new| 2010-04-01
> 15:45:08|12726|10:10|new|new.pm:reload_image(670)|esx3-basewinxp-v0
> > failed to load on esx3-basewinxp-v0, returning
> > 932 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 933 |12726|10:10|new| (-1) new.pm, reload_image (line: 670)
> > 934 |12726|10:10|new| (-2) new.pm, process (line: 266)
> > 935 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
> > 936 |12726|10:10|new| (-4) vcld, main (line: 341)
> > 937
> > 938 2010-04-01 15:45:08|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, loadimagefailed, esx3-basewinxp-v0 failed to load on
> > esx3-basewinxp-v0
> > 939
> > 940 |12726|10:10|new| ---- WARNING ----
> > 941 |12726|10:10|new| 2010-04-01
> 15:45:08|12726|10:10|new|new.pm:process(313)|failed
> > to load esx3-basewinxp-v0 with esx3-basewinxp-v0
> > 942 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 943 |12726|10:10|new| (-1) new.pm, process (line: 313)
> > 944 |12726|10:10|new| (-2) vcld, make_new_child (line: 594)
> > 945 |12726|10:10|new| (-3) vcld, main (line: 341)
> > 946
> > 947 2010-04-01
> > 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> > to retrieve current state of computer esx3-basewinxp-v0 from the database
> > 948 2010-04-01
> > 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> > current state of computer esx3-basewinxp-v0 from the database: reloading
> > 949 2010-04-01
> > 15:45:08|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> > updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> > 950 |12726|10:10|new| computer_state_name = reloading
> > 951 No recipient addresses found in header
> > 952 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
> > Sending mail To: , PROBLEM -- State.pm
> > 953
> > 954 |12726|10:10|new| ---- CRITICAL ----
> > 955 |12726|10:10|new| 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(290)|reservation failed
> > on esx3-basewinxp-v0: process failed after trying to load or make available
> > 956 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> > 957 |12726|10:10|new| (-1) State.pm, reservation_failed (line: 290)
> > 958 |12726|10:10|new| (-2) new.pm, process (line: 316)
> > 959 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
> > 960 |12726|10:10|new| (-4) vcld, main (line: 341)
> > 961
> > 962 2010-04-01 15:45:08|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> > computer=3, failed, process failed after trying to load or make available
> > 963 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(293)|inserted
> > computerloadlog entry
> > 964 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(301)|updated log ending
> > value to 'failed', logid=11
> > 965 2010-04-01
> 15:45:08|12726|10:10|new|utils.pm:update_computer_state(2228)|computer
> > 3 state updated to: failed
> > 966 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(312)|computer
> > esx3-basewinxp-v0 (3) state set to failed
> > 967 2010-04-01 15:45:08|12726|10:10|new|utils.pm:update_request_state(2186)|request
> > 10 state updated to: failed, laststate to: new
> > 968 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(325)|set request state
> > to 'failed'/'new'
> > 969 2010-04-01 15:45:08|12726|10:10|new|utils.pm:is_inblockrequest(6972)|zero
> > rows were returned from database select
> > 970 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(343)|esx3-basewinxp-v0
> > is NOT in blockcomputers table
> > 971 2010-04-01
> > 15:45:08|12726|10:10|new|State.pm:reservation_failed(346)|exiting 1
> > 972 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(905)|destructor
> > called, ref($self)=VCL::new
> > 973 2010-04-01
> 15:45:08|12726|10:10|new|utils.pm:delete_computerloadlog_reservation(7551)|removing
> > computerloadlog entries matching loadstate = begin
> > 974 2010-04-01
> 15:45:08|12726|10:10|new|utils.pm:delete_computerloadlog_reservation(7598)|deleted
> > rows from computerloadlog for reservation id=10
> > 975 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(912)|removed
> > computerloadlog rows with loadstate=begin for reservation
> > 976 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(924)|number of
> > database handles state process created: 1
> > 977 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(933)|process
> > has a database handle stored in $ENV{dbh}, attempting disconnect
> > 978 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(935)|$ENV{dbh}:
> > database disconnect successful
> > 979 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(949)|VCL::new
> > process 12726 exiting
> > 980 2010-04-01 15:45:08|5754|vcld:REAPER(744)|VCL process exited for
> > reservation 10
> > 981 2010-04-01 15:45:08|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:45:08
> > 982 2010-04-01 15:45:13|5754|vcld:main(165)|lastcheckin time updated for
> > management node 1: 2010-04-01 15:45:13
> >
> >
> ================================================================
> =====================
> >
> > Thank you,
> > Amit
> >
> >
> >
Re: [Spam: 6.0] Re: VCL Getting Reservation to work
Posted by Ryan Johnson <rj...@gwmail.gwu.edu>.
Yea, that is a little hard on the eyes, any chance you could send screen
shots in a view where the columns are lined up with the column names?
On Thu, Apr 1, 2010 at 4:00 PM, Kumar, Amit H. <AH...@odu.edu> wrote:
> The Savior! Thank you Ryan getting back!!!
>
> Not sure if this is the best format for readability...please let me know
> otherwise.
>
> I am also a recent log that I tried with verbose ssh enabled etc, I haven't
> looked at it yet. It went past my previous log. .....................
>
> Image: Table: Rows: 10
> ====================
> id name prettyname ownerid platformid OSid imagemetaid minram
> minprocnumber minprocspeed minnetwork maxconcurrent reloadtime deleted
> test lastupdate forcheckout maxinitialtime project size architecture
> description usage basedoffrevisionid
> 1 winxp-base1-v0 No Apps (WinXP) 1 [->admin] 1 [->i386] 7 [->winxp] NULL 0
> 1 0 10 NULL 14 0 0 2007-04-11 16:07:38 1 0 vcl 1045 x86 NULL NULL 0
> 2 rhel4-base2-v0 Red Hat Enterprise Linux 4.4 Base (KS) 1 [->admin] 1
> [->i386] 12 [->rhel4] NULL 1024 1 1024 100 NULL 13 0 0 2007-03-02 16:33:33 1
> 0 vcl 0 x86 NULL NULL 0
> 3 rh4image-base3-v0 RHEL4 base (image) 1 [->admin] 1 [->i386] 13
> [->rh4image] NULL 10 1 1024 100 NULL 10 0 0 2007-01-24 15:02:07 1 0 vcl 1450
> x86 NULL NULL 0
> 4 noimage No Image 1 [->admin] 1 [->i386] 2 [->sun4x_58] NULL 0 1 0 10 NULL
> 0 0 0 NULL 1 0 vcl 1450 x86 NULL NULL 0
> 5 rhfc5-fc5base5-v0 Red Hat Fedora Core 5 base (KS) 1 [->admin] 1 [->i386]
> 15 [->rhfc5] NULL 1024 1 1024 100 NULL 13 0 0 2006-10-02 10:04:24 1 0 vcl
> 1450 x86 NULL NULL 0
> 6 fc5image-FC5baseRHFC56-v0 FC5base(RHFC5) 1 [->admin] 1 [->i386] 14
> [->fc5image] NULL 64 1 500 10 NULL 10 0 0 2006-10-02 14:13:26 1 0 vcl 0 x86
> NULL NULL 0
> 7 vmwarewinxp-base7-v1 No Apps (WinXP vmware) 1 [->admin] 1 [->i386] 16
> [->vmwarewinxp] NULL 512 1 1024 100 NULL 5 0 0 2007-04-04 09:45:38 1 0 vcl
> 3244 x86 NULL NULL 0
> 8 rh4image-VMwareserverhostRHEL48-v3 VMware server host (RHEL4) 1 [->admin]
> 1 [->i386] 13 [->rh4image] NULL 64 1 500 10 NULL 10 0 0 2007-11-26 11:23:53
> 1 0 vcl 1450 x86 NULL NULL 0
> 9 esx35-base-v0 VMware ESX 3.5 standard server 1 [->admin] 1 [->i386] 20
> [->esx35] NULL 2048 2 2000 100 NULL 9 0 0 2008-03-24 14:23:54 1 0 vcl 1450
> x86 NULL NULL 0
> 10 esx3-basewinxp-v0 No Apps (ESXi WinXP) 1 [->admin] 1 [->i386] 20
> [->esx35] NULL 512 1 1024 100 NULL 9 0 0 2010-03-24 17:35:12 1 0 vcl 4100
> x86 NULL NULL 0
> /End Image Table====================
>
>
> ImageRevision Table: Rows: 11
> ===========================
> id imageid revision userid datecreated deleted datedeleted
> production comments imagename
> 1 1 [->No Apps (WinXP)] 0 1 [->admin] 2007-03-01 14:46:26 0 NULL 1 NULL
> winxp-base1-v0
> 2 2 [->Red Hat Enterprise Linux 4.4 Base (KS)] 0 1 [->admin] 2006-09-22
> 16:33:24 0 NULL 1 NULL rhel4-base2-v0
> 3 3 [->RHEL4 base (image)] 0 1 [->admin] 2007-01-24 15:04:11 0 NULL 1 NULL
> rh4image-base3-v0
> 4 4 [->No Image] 0 1 [->admin] 1980-01-01 00:00:00 0 NULL 1 NULL noimage
> 5 5 [->Red Hat Fedora Core 5 base (KS)] 0 1 [->admin] 2006-10-02 10:04:24 0
> NULL 1 NULL rhfc5-fc5base5-v0
> 6 6 [->FC5base(RHFC5)] 0 1 [->admin] 2006-10-02 14:13:26 0 NULL 1 NULL
> fc5image-FC5baseRHFC56-v0
> 7 7 [->No Apps (WinXP vmware)] 0 1 [->admin] 2007-04-03 11:24:12 0 NULL 1
> NULL vmwarewinxp-base7-v0
> 8 8 [->VMware server host (RHEL4)] 0 1 [->admin] 2007-01-30 10:50:56 0 NULL
> 1 NULL rh4image-VMwareserverhostRHEL48-v0
> 9 9 [->VMware ESX 3.5 standard server] 0 1 [->admin] 2008-03-24 14:23:54 0
> NULL 1 NULL esx35-base-v0
> 10 7 [->No Apps (WinXP vmware)] 1 1 [->admin] 2010-03-18 14:56:50 0 NULL 1
> NULL vmwarewinxp-base7-v1
> 11 10 [->No Apps (ESXi WinXP)] 0 1 [->admin] 2010-03-24 17:35:12 0 NULL 1
> NULL esx3-basewinxp-v0
> /End ImageRevision Table====================
>
>
> Computer Table: Rows: 3
> ============================================
> id stateid ownerid platformid scheduleid currentimageid
> preferredimageid nextimageid imagerevisionid RAM procnumber procspeed
> network hostname IPaddress privateIPaddress eth0macaddress
> eth1macaddress type provisioningid drivetype deleted notes lastcheck
> location dsa dsapub rsa rsapub host hostpub vmhostid vmtypeid
> 1 10 [->maintenance] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0
> 0 0 512 2 2200 1000 resxi-2.odu.edu 128.82.128.218 NULL NULL NULL
> virtualmachine 4 [->vmware_esx] hda 1 NULL NULL NULL NULL NULL NULL NULL
> [BLOB - 0B] NULL NULL NULL
> 2 2 [->available] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0
> 0 20480 8 2200 1000 esx35-hypervisor-1.odu.edu 128.82.128.252 NULL NULL
> NULL blade 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL [BLOB -
> 0B] NULL NULL NULL
> 3 5 [->failed] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0 0
> 512 2 2200 100 esx3-basewinxp-v0.odu.edu 128.82.128.217 NULL NULL NULL
> virtualmachine 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL [BLOB
> - 0B] NULL 1 NULL
> /End Computer Table======================================
>
>
> Vmprofile Table: Rows: 4
> ========================
> id profilename vmtypeid imageid nasshare datastorepath vmpath
> virtualswitch0 virtualswitch1 vmdisk username password
> 1 VMware GSX standard 3 [->vmwareGSX] 8 [->VMware server host (RHEL4)] NULL
> /var/lib/vmware/Virtual Machines NULL VMnet0 VMnet2 localdisk NULL NULL
> 2 Vmware ESX standard network mounted share 5 [->vmwareESX3] 9 [->VMware
> ESX 3.5 standard server] NULL /vmfs/volumes/nfs1 /vmfs/volumes/storage1 VM
> Network Virtual Machine Public Network networkdisk NULL NULL
> 3 Vmware ESX standard localdisk 6 [->vmwareESXi] 9 [->VMware ESX 3.5
> standard server] NULL 192.168.129.37:/install/image/vmware_images
> 192.168.129.37:/install/image/vmware_images VM Network Private Vlan328
> localdisk root
> 4 Vmware ESX SAN 5 [->vmwareESX3] 9 [->VMware ESX 3.5 standard server] NULL
> /vmfs/volumes/NetApp /vmfs/volumes/storage1 Intranet2 MCNC Public
> networkdisk NULL NULL
> /End Vmprofile===============================
>
> VMhost Table: Rows: 1
> =====================
> id computerid vmlimit vmprofileid vmkernalnic vmwaredisk
> 1 2 [->esx35-hypervisor-1.odu.edu] 2 3 NULL localdisk
> /End Vmhost Table==========================
>
>
> ================================= LOG FILE
> ===========================================
> 13 2010-04-01 15:37:19|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:37:19
> 14 2010-04-01 15:37:24|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:37:24
> 15 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:reservation_being_processed(9634)|computerloadlog
> 'begin' entry does NOT exist for reservation 9
> 16 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:reservation_being_processed(9686)|reservation
> is NOT currently being processed
> 17 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(276)|reservation 9 is
> NOT already being processed
> 18
> 19 |5754|9:9|deleted| ---- WARNING ----
> 20 |5754|9:9|deleted| 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:get_request_info(5218)|preferredimageid
> is not set for computer id=2
> 21 |5754|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 22 |5754|9:9|deleted| (-1) utils.pm, get_request_info (line: 5218)
> 23 |5754|9:9|deleted| (-2) vcld, main (line: 280)
> 24
> 25 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:get_request_info(5354)|standalone
> affiliation found: Local
> 26 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:get_management_node_info(6815)|management
> node info retrieved from database for colbert
> 27 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(281)|retrieved
> request information from database
> 28 2010-04-01
> 15:37:24|5754|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 9
> 29 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:update_request_state(2186)|request
> 9 state updated to: pending, laststate to: deleted
> 30 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> entry not inserted, reservation has been deleted
> 31 2010-04-01
> 15:37:24|5754|9:9|deleted|vcld:make_new_child(489)|request will be processed
> by reclaim.pm
> 32 2010-04-01 15:37:24|5754|9:9|deleted|vcld:make_new_child(509)|loaded
> VCL::reclaim module
> 33 2010-04-01 15:37:24|12719|9:9|deleted|vcld:make_new_child(582)|vcld
> environment variable set to 0 for this process
> 34 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> called, class=VCL::reclaim
> 35 2010-04-01
> 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::reclaim object created
> 36 2010-04-01
> 15:37:24|5754|9:9|deleted|vcld:make_new_child(566)|current number of forked
> kids: 1
> 37 2010-04-01
> 15:37:24|12719|9:9|deleted|State.pm:initialize(85)|obtained a database
> handle for this state process, stored as $ENV{dbh}
> 38 2010-04-01
> 15:37:24|12719|9:9|deleted|State.pm:check_image_os(820)|no corrections need
> to be made, not an imaging request, returning 1
> 39 2010-04-01
> 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 9
> 40 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7863)|reservation
> count: 1
> 41 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7882)|PARENTIMAGE:
> 1
> 42 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7883)|SUBIMAGE:
> 0
> 43 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7901)|renamed
> process to 'vcld VCL::reclaim 9:9 deleted'
> 44 2010-04-01
> 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 9
> 45 2010-04-01
> 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 9
> 46 2010-04-01
> 15:37:24|12719|9:9|deleted|State.pm:initialize(121)|attempting to load
> provisioning module: VCL::Module::Provisioning::esx
> 47 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(138)|esx
> vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> 48 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(140)|vmware
> ESX module initialized
> 49 2010-04-01
> 15:37:24|12719|9:9|deleted|State.pm:initialize(127)|VCL::Module::Provisioning::esx
> module loaded
> 50 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> called, class=VCL::Module::Provisioning::esx
> 51 2010-04-01
> 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::Provisioning::esx
> object created
> 52 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(138)|esx
> vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> 53 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(140)|vmware
> ESX module initialized
> 54 2010-04-01
> 15:37:24|12719|9:9|deleted|State.pm:initialize(131)|VCL::Module::Provisioning::esx
> provisioner object created
> 55 2010-04-01
> 15:37:24|12719|9:9|deleted|State.pm:initialize(145)|attempting to load OS
> module: VCL::Module::OS::Linux
> 56 2010-04-01
> 15:37:24|12719|9:9|deleted|State.pm:initialize(151)|VCL::Module::OS::Linux
> module loaded
> 57 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor
> called, class=VCL::Module::OS::Linux
> 58 2010-04-01
> 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::OS::Linux object
> created
> 59 2010-04-01
> 15:37:24|12719|9:9|deleted|State.pm:initialize(154)|VCL::Module::OS::Linux
> OS object created
> 60 2010-04-01
> 15:37:24|12719|9:9|deleted|State.pm:initialize(169)|returning 1
> 61 2010-04-01
> 15:37:24|12719|9:9|deleted|vcld:make_new_child(591)|VCL::reclaim object
> created and initialized
> 62 2010-04-01
> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation
> remote IP is not defined
> 63 2010-04-01
> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1946)|attempting
> to retrieve current state of computer esx35-hypervisor-1 from the database
> 64 2010-04-01
> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1977)|retrieved
> current state of computer esx35-hypervisor-1 from the database: available
> 65 2010-04-01
> 15:37:24|12719|9:9|deleted|DataStructure.pm:_automethod(697)|data structure
> updated: $self->request_data->{reservation}{9}{computer}{state}{name}
> 66 |12719|9:9|deleted| computer_state_name = available
> 67 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> entry not inserted, reservation has been deleted
> 68 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> entry not inserted, reservation has been deleted
> 69 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> entry not inserted, reservation has been deleted
> 70 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> entry not inserted, reservation has been deleted
> 71 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> entry not inserted, reservation has been deleted
> 72 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(138)|computer
> type is blade
> 73 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(142)|request
> laststate is reserved, attempting to sanitize computer
> 74 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(172)|calling
> VCL::Module::OS::Linux::sanitize() subroutine
> 75 2010-04-01
> 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation
> remote IP is not defined
> 76 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> SSH command on esx35-hypervisor-1:
> 77 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'netstat -an' 2>&1
> 78
> 79 |12719|9:9|deleted| ---- WARNING ----
> 80 |12719|9:9|deleted| 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 1/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
> status: 255, SSH exits with the exit status of the remote command or with
> 255 if an error occurred, output:
> 81 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> found
> 82 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 83 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 84 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> 85 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> 86 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 87 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 88 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 89
> 90 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 2 seconds before making next SSH attempt
> 91 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 2/3: executing SSH command on esx35-hypervisor-1:
> 92 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'netstat -an' 2>&1
> 93
> 94 |12719|9:9|deleted| ---- WARNING ----
> 95 |12719|9:9|deleted| 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 2/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
> status: 255, SSH exits with the exit status of the remote command or with
> 255 if an error occurred, output:
> 96 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> found
> 97 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 98 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 99 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> 100 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> 101 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 102 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 103 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 104
> 105 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 15 seconds before making next SSH attempt
> 106 2010-04-01 15:37:29|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:37:29
> 107 2010-04-01 15:37:34|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:37:34
> 108 2010-04-01 15:37:39|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:37:39
> 109 2010-04-01 15:37:39|5754|10:10|new|utils.pm:reservation_being_processed(9634)|computerloadlog
> 'begin' entry does NOT exist for reservation 10
> 110 2010-04-01 15:37:39|5754|10:10|new|utils.pm:reservation_being_processed(9686)|reservation
> is NOT currently being processed
> 111 2010-04-01 15:37:39|5754|10:10|new|vcld:main(276)|reservation 10 is
> NOT already being processed
> 112
> 113 |5754|10:10|new| ---- WARNING ----
> 114 |5754|10:10|new| 2010-04-01 15:37:39|5754|10:10|new|utils.pm:get_request_info(5218)|preferredimageid
> is not set for computer id=3
> 115 |5754|10:10|new| ( 0) utils.pm, notify (line: 737)
> 116 |5754|10:10|new| (-1) utils.pm, get_request_info (line: 5218)
> 117 |5754|10:10|new| (-2) vcld, main (line: 280)
> 118
> 119 2010-04-01 15:37:39|5754|10:10|new|utils.pm:get_request_info(5354)|standalone
> affiliation found: Local
> 120 2010-04-01 15:37:39|5754|10:10|new|utils.pm:get_management_node_info(6815)|management
> node info retrieved from database for colbert
> 121 2010-04-01 15:37:39|5754|10:10|new|vcld:main(281)|retrieved request
> information from database
> 122 2010-04-01
> 15:37:39|5754|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 10
> 123 2010-04-01 15:37:39|5754|10:10|new|utils.pm:update_request_state(2186)|request
> 10 state updated to: pending, laststate to: new
> 124 2010-04-01 15:37:39|5754|10:10|new|utils.pm:insertloadlog(4710)|inserted
> computer=3, begin, beginning to process, state is new
> 125 2010-04-01 15:37:39|5754|10:10|new|vcld:make_new_child(509)|loaded
> VCL::new module
> 126 2010-04-01 15:37:39|12726|10:10|new|vcld:make_new_child(582)|vcld
> environment variable set to 0 for this process
> 127 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> called, class=VCL::new
> 128 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::new
> object created
> 129 2010-04-01 15:37:39|5754|10:10|new|vcld:make_new_child(566)|current
> number of forked kids: 2
> 130 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(85)|obtained
> a database handle for this state process, stored as $ENV{dbh}
> 131 2010-04-01 15:37:39|12726|10:10|new|State.pm:check_image_os(820)|no
> corrections need to be made, not an imaging request, returning 1
> 132 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 10
> 133 2010-04-01 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7863)|reservation
> count: 1
> 134 2010-04-01 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7882)|PARENTIMAGE:
> 1
> 135 2010-04-01 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7883)|SUBIMAGE:
> 0
> 136 2010-04-01 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7901)|renamed
> process to 'vcld VCL::new 10:10 new'
> 137 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 10
> 138 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 10
> 139 2010-04-01
> 15:37:39|12726|10:10|new|State.pm:initialize(121)|attempting to load
> provisioning module: VCL::Module::Provisioning::esx
> 140 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx
> vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> 141 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(140)|vmware
> ESX module initialized
> 142 2010-04-01
> 15:37:39|12726|10:10|new|State.pm:initialize(127)|VCL::Module::Provisioning::esx
> module loaded
> 143 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> called, class=VCL::Module::Provisioning::esx
> 144 2010-04-01
> 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::Provisioning::esx
> object created
> 145 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx
> vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> 146 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(140)|vmware
> ESX module initialized
> 147 2010-04-01
> 15:37:39|12726|10:10|new|State.pm:initialize(131)|VCL::Module::Provisioning::esx
> provisioner object created
> 148 2010-04-01
> 15:37:39|12726|10:10|new|State.pm:initialize(145)|attempting to load OS
> module: VCL::Module::OS::Linux
> 149 2010-04-01
> 15:37:39|12726|10:10|new|State.pm:initialize(151)|VCL::Module::OS::Linux
> module loaded
> 150 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor
> called, class=VCL::Module::OS::Linux
> 151 2010-04-01
> 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::OS::Linux object
> created
> 152 2010-04-01
> 15:37:39|12726|10:10|new|State.pm:initialize(154)|VCL::Module::OS::Linux OS
> object created
> 153 2010-04-01
> 15:37:39|12726|10:10|new|State.pm:initialize(169)|returning 1
> 154 2010-04-01
> 15:37:39|12726|10:10|new|vcld:make_new_child(591)|VCL::new object created
> and initialized
> 155 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 10
> 156 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> to retrieve current state of computer esx3-basewinxp-v0 from the database
> 157 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> current state of computer esx3-basewinxp-v0 from the database: available
> 158 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> 159 |12726|10:10|new| computer_state_name = available
> 160
> 161 |12726|10:10|new| ---- WARNING ----
> 162 |12726|10:10|new| 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> data has not been initialized for get_computer_preferredimage_id:
> $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> 163 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 164 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> 165 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 166 |12726|10:10|new| (-3) new.pm, process (line: 106)
> 167 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> 168 |12726|10:10|new| (-5) vcld, main (line: 341)
> 169
> 170
> 171 |12726|10:10|new| ---- WARNING ----
> 172 |12726|10:10|new| 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> data has not been initialized for get_computer_preferredimage_name:
> $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> 173 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 174 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> 175 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 176 |12726|10:10|new| (-3) new.pm, process (line: 107)
> 177 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> 178 |12726|10:10|new| (-5) vcld, main (line: 341)
> 179
> 180 2010-04-01 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
> variable: 1
> 181 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(134)|reservation
> is parent = 1
> 182 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(135)|preload
> only = 0
> 183 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(136)|originating
> request state = new
> 184 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(137)|originating
> request laststate = new
> 185 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(138)|originating
> computer state = available
> 186 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(139)|originating
> computer type = virtualmachine
> 187 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 10
> 188 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> to retrieve current state of computer esx3-basewinxp-v0 from the database
> 189 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> current state of computer esx3-basewinxp-v0 from the database: available
> 190 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> 191 |12726|10:10|new| computer_state_name = available
> 192
> 193 |12726|10:10|new| ---- WARNING ----
> 194 |12726|10:10|new| 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> data has not been initialized for get_computer_preferredimage_id:
> $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> 195 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 196 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> 197 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 198 |12726|10:10|new| (-3) new.pm, computer_not_being_used (line: 750)
> 199 |12726|10:10|new| (-4) new.pm, process (line: 170)
> 200 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> 201 |12726|10:10|new| (-6) vcld, main (line: 341)
> 202
> 203
> 204 |12726|10:10|new| ---- WARNING ----
> 205 |12726|10:10|new| 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> data has not been initialized for get_computer_preferredimage_name:
> $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> 206 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 207 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> 208 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 209 |12726|10:10|new| (-3) new.pm, computer_not_being_used (line: 751)
> 210 |12726|10:10|new| (-4) new.pm, process (line: 170)
> 211 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> 212 |12726|10:10|new| (-6) vcld, main (line: 341)
> 213
> 214 2010-04-01 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
> variable: 1
> 215 2010-04-01 15:37:39|12726|10:10|new|new.pm:computer_not_being_used(788)|esx3-basewinxp-v0
> state is available
> 216 2010-04-01 15:37:39|12726|10:10|new|new.pm:computer_not_being_used(798)|esx3-basewinxp-v0
> is available, its state is available
> 217 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(170)|esx3-basewinxp-v0
> is not being used
> 218 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 10
> 219 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> to retrieve current state of computer esx3-basewinxp-v0 from the database
> 220 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> current state of computer esx3-basewinxp-v0 from the database: available
> 221 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> 222 |12726|10:10|new| computer_state_name = available
> 223
> 224 |12726|10:10|new| ---- WARNING ----
> 225 |12726|10:10|new| 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> data has not been initialized for get_computer_preferredimage_id:
> $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
> 226 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 227 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> 228 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 229 |12726|10:10|new| (-3) new.pm, reload_image (line: 480)
> 230 |12726|10:10|new| (-4) new.pm, process (line: 266)
> 231 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> 232 |12726|10:10|new| (-6) vcld, main (line: 341)
> 233
> 234
> 235 |12726|10:10|new| ---- WARNING ----
> 236 |12726|10:10|new| 2010-04-01
> 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> data has not been initialized for get_computer_preferredimage_name:
> $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
> 237 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 238 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> 239 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 240 |12726|10:10|new| (-3) new.pm, reload_image (line: 481)
> 241 |12726|10:10|new| (-4) new.pm, process (line: 266)
> 242 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> 243 |12726|10:10|new| (-6) vcld, main (line: 341)
> 244
> 245 2010-04-01 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment
> variable: 1
> 246 2010-04-01 15:37:39|12726|10:10|new|new.pm:reload_image(514)|calling
> VCL::Module::Provisioning::esx->node_status()
> 247 2010-04-01 15:37:39|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> computer=3, statuscheck, checking status of node
> 248 $VAR1 = bless( {
> 249 'data' => bless( do{\(my $o = 1)},
> 'VCL::DataStructure' ),
> 250 'os' => bless( {
> 251 'provisioner' => $VAR1,
> 252 'data' => $VAR1->{'data'}
> 253 }, 'VCL::Module::OS::Linux' )
> 254 }, 'VCL::Module::Provisioning::esx' );
> 255 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(787)|Entering
> node_status, checking status of esx3-basewinxp-v0
> 256 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(788)|request_for_imaging:
> 0
> 257 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(789)|requeseted
> image name: esx3-basewinxp-v0
> 258 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(819)|checking
> if esx3-basewinxp-v0 is pingable
> 259 2010-04-01 15:37:40|12726|10:10|new|esx.pm:node_status(825)|esx3-basewinxp-v0
> is not pingable (0)
> 260 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(560)|node_status
> returned a scalar: RELOAD
> 261 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(602)|node
> status is RELOAD, esx3-basewinxp-v0 will be reloaded
> 262 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> computer=3, loadimageblade, esx3-basewinxp-v0 must be reloaded with
> esx3-basewinxp-v0
> 263 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(616)|calling
> VCL::Module::Provisioning::esx->does_image_exist()
> 264 2010-04-01 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> SSH command on 192.168.129.37:
> 265 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> 192.168.129.37 'ls -1 /install/image/vmware_images/golden 2>&1' 2>&1
> 266 2010-04-01 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> output:
> 267 |12726|10:10|new| esx3-basewinxp-v0
> 268 2010-04-01 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> command executed on 192.168.129.37, returning (0, "esx3-basewinxp-v0")
> 269 2010-04-01 15:37:40|12726|10:10|new|esx.pm:does_image_exist(917)|image
> esx3-basewinxp-v0 exists
> 270 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(619)|esx3-basewinxp-v0
> exists on this management node
> 271 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> computer=3, doesimageexists, confirmed image exists
> 272 2010-04-01 15:37:40|12726|10:10|new|utils.pm:update_computer_state(2228)|computer
> 3 state updated to: reloading
> 273 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(652)|computer
> esx3-basewinxp-v0 state set to reloading
> 274 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> computer=3, info, computer state updated to reloading
> 275 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(663)|calling
> VCL::Module::Provisioning::esx->load() subroutine
> 276 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> computer=3, info, calling VCL::Module::Provisioning::esx->load() subroutine
> 277 2010-04-01 15:37:40|12726|10:10|new|esx.pm:
> load(165)|****************************************************
> 278
> 279 |12726|10:10|new| ---- WARNING ----
> 280 |12726|10:10|new| 2010-04-01
> 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> data has not been initialized for get_computer_eth0_mac_address:
> $self->request_data->{reservation}{10}{computer}{eth0macaddress}
> 281 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 282 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> 283 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 284 |12726|10:10|new| (-3) esx.pm, load (line: 181)
> 285 |12726|10:10|new| (-4) new.pm, reload_image (line: 665)
> 286 |12726|10:10|new| (-5) new.pm, process (line: 266)
> 287 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
> 288
> 289
> 290 |12726|10:10|new| ---- WARNING ----
> 291 |12726|10:10|new| 2010-04-01
> 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding
> data has not been initialized for get_computer_eth1_mac_address:
> $self->request_data->{reservation}{10}{computer}{eth1macaddress}
> 292 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 293 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
> 294 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 295 |12726|10:10|new| (-3) esx.pm, load (line: 182)
> 296 |12726|10:10|new| (-4) new.pm, reload_image (line: 665)
> 297 |12726|10:10|new| (-5) new.pm, process (line: 266)
> 298 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
> 299
> 300 2010-04-01 15:37:40|12726|10:10|new|esx.pm:load(193)|Calling
> get_vmware_host_info
> 301 No recipient addresses found in header
> 302 2010-04-01 15:37:41|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
> Sending mail To: , PROBLEM -- esx.pm
> 303
> 304 |12726|10:10|new| ---- CRITICAL ----
> 305 |12726|10:10|new| 2010-04-01 15:37:40|12726|10:10|new|esx.pm:get_vmware_host_info(1012)|AMIT
> DEBUGGING: esx35-hypervisor-1.odu.edu
> 306 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 307 |12726|10:10|new| (-1) esx.pm, get_vmware_host_info (line: 1012)
> 308 |12726|10:10|new| (-2) esx.pm, load (line: 194)
> 309 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
> 310 |12726|10:10|new| (-4) new.pm, process (line: 266)
> 311 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> 312 |12726|10:10|new| (-6) vcld, main (line: 341)
> 313
> 314 2010-04-01 15:37:41|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 3/3: executing SSH command on esx35-hypervisor-1:
> 315 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'netstat -an' 2>&1
> 316
> 317 |12719|9:9|deleted| ---- WARNING ----
> 318 |12719|9:9|deleted| 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 3/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit
> status: 255, SSH exits with the exit status of the remote command or with
> 255 if an error occurred, output:
> 319 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> found
> 320 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 321 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 322 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> 323 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> 324 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 325 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 326 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 327
> 328
> 329 |12719|9:9|deleted| ---- WARNING ----
> 330 |12719|9:9|deleted| 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'netstat -an' 2>&1,
> exit status: 255, output:
> 331 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not
> found
> 332 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 333 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
> 334 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
> 335 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
> 336 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 337 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 338 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 339
> 340 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> SSH command on esx35-hypervisor-1:
> 341 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> /etc/ssh/external_sshd_config"' 2>&1
> 342
> 343 |12719|9:9|deleted| ---- WARNING ----
> 344 |12719|9:9|deleted| 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 1/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx
> "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255, SSH
> exits with the exit status of the remote command or with 255 if an error
> occurred, output:
> 345 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not
> found
> 346 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 347 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 348 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line: 913)
> 349 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
> 350 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
> 351 |12719|9:9|deleted| (-5) reclaim.pm, process (line: 173)
> 352 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
> 353
> 354 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 2 seconds before making next SSH attempt
> 355 2010-04-01 15:37:43|12726|10:10|new|esx.pm:get_vmware_host_info(1019)|host
> info output for esx35-hypervisor-1
> 356 |12726|10:10|new| Host Information
> 357 |12726|10:10|new| Host Name: esx35-hypervisor-1.odu.edu
> 358 2010-04-01 15:37:43|12726|10:10|new|esx.pm:get_vmware_host_info(1025)|found
> hostname_value= esx35-hypervisor-1.odu.edu
> 359 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(197)|Collected
> esx35-hypervisor-1.odu.edu for vmware host name
> 360 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(210)|DATASTORE IP
> is 192.168.129.37 and DATASTORE_SHARE_PATH is /install/image/vmware_images
> 361 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(211)|Entered ESX
> module, loading esx3-basewinxp-v0 on esx3-basewinxp-v0 (on
> esx35-hypervisor-1.odu.edu) for reservation 10
> 362 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(212)|Datastore:
> 192.168.129.37:/install/image/vmware_images
> 363 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(223)|VM info
> command: /usr/lib/vmware-vcli/apps/vm/vminfo.pl --server
> 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --username root --password
> 'xxxxxxxxxxx'
> 364 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 2/3: executing SSH command on esx35-hypervisor-1:
> 365 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> /etc/ssh/external_sshd_config"' 2>&1
> 366 2010-04-01 15:37:44|12726|10:10|new|esx.pm:load(226)|VM info
> output: Virtual Machine esx3-basewinxp-v0 not found.
> 367 2010-04-01 15:37:44|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> SSH command on 192.168.129.37:
> 368 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> 192.168.129.37 'rm -rf /install/image/vmware_images/inuse/esx3-basewinxp-v0'
> 2>&1
> 369 2010-04-01 15:37:44|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:37:44
> 370
> 371 |12719|9:9|deleted| ---- WARNING ----
> 372 |12719|9:9|deleted| 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 2/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx
> "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255, SSH
> exits with the exit status of the remote command or with 255 if an error
> occurred, output:
> 373 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not
> found
> 374 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 375 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 376 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line: 913)
> 377 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
> 378 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
> 379 |12719|9:9|deleted| (-5) reclaim.pm, process (line: 173)
> 380 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
> 381
> 382 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 15 seconds before making next SSH attempt
> 383 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> output:
> 384 |12726|10:10|new| none
> 385 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> command executed on 192.168.129.37, returning (0, "none")
> 386 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(262)|Removed old vm
> folder
> 387 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> SSH command on 192.168.129.37:
> 388 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> 192.168.129.37 'mkdir /install/image/vmware_images/inuse/esx3-basewinxp-v0'
> 2>&1
> 389 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> output:
> 390 |12726|10:10|new| none
> 391 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> command executed on 192.168.129.37, returning (0, "none")
> 392 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> SSH command on 192.168.129.37:
> 393 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> 192.168.129.37 'cp
> /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk
> /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk'
> 2>&1
> 394 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> output:
> 395 |12726|10:10|new| none
> 396 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> command executed on 192.168.129.37, returning (0, "none")
> 397 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(278)|COPIED VMDK
> SUCCESSFULLY
> 398 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> SSH command on 192.168.129.37:
> 399 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> 192.168.129.37 'cp
> /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk
> /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk'
> 2>&1
> 400 2010-04-01 15:37:49|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:37:49
> 401 2010-04-01 15:37:54|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:37:54
> 402 2010-04-01 15:37:59|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 3/3: executing SSH command on esx35-hypervisor-1:
> 403 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> /etc/ssh/external_sshd_config"' 2>&1
> 404 2010-04-01 15:37:59|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:37:59
> 405 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6262)|run_ssh_command
> output:
> 406 |12719|9:9|deleted| ash: pkill: not found
> 407 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6272)|SSH
> command executed on esx35-hypervisor-1, command:
> 408 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f
> /etc/ssh/external_sshd_config"' 2>&1
> 409 |12719|9:9|deleted| returning (127, "ash: pkill: not found")
> 410 2010-04-01
> 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(916)|output detected:
> ash: pkill: not found
> 411 2010-04-01
> 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(921)|ext_sshd on
> esx35-hypervisor-1 stopped
> 412 2010-04-01
> 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(884)|stopped external sshd
> 413 2010-04-01
> 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(888)|access has been
> revoked to esx35-hypervisor-1
> 414 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> SSH command on esx35-hypervisor-1:
> 415 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> 416
> 417 |12719|9:9|deleted| ---- WARNING ----
> 418 |12719|9:9|deleted| 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 1/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> admin, exit status: 255, SSH exits with the exit status of the remote
> command or with 255 if an error occurred, output:
> 419 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/passwd'
> 420 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/shadow'
> 421 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/group'
> 422 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/gshadow'
> 423 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 424 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 425 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> 426 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 427 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 428 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 429 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 430
> 431 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 2 seconds before making next SSH attempt
> 432 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 2/3: executing SSH command on esx35-hypervisor-1:
> 433 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> 434
> 435 |12719|9:9|deleted| ---- WARNING ----
> 436 |12719|9:9|deleted| 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 2/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> admin, exit status: 255, SSH exits with the exit status of the remote
> command or with 255 if an error occurred, output:
> 437 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/passwd'
> 438 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/shadow'
> 439 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/group'
> 440 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/gshadow'
> 441 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 442 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 443 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> 444 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 445 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 446 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 447 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 448
> 449 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 15 seconds before making next SSH attempt
> 450 2010-04-01 15:38:04|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:38:04
> 451 2010-04-01 15:38:09|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:38:09
> 452 2010-04-01 15:38:14|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:38:14
> 453 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 3/3: executing SSH command on esx35-hypervisor-1:
> 454 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> 455
> 456 |12719|9:9|deleted| ---- WARNING ----
> 457 |12719|9:9|deleted| 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 3/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> admin, exit status: 255, SSH exits with the exit status of the remote
> command or with 255 if an error occurred, output:
> 458 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/passwd'
> 459 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/shadow'
> 460 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/group'
> 461 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/gshadow'
> 462 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 463 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 464 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> 465 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 466 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 467 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 468 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 469
> 470
> 471 |12719|9:9|deleted| ---- WARNING ----
> 472 |12719|9:9|deleted| 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel
> admin' 2>&1, exit status: 255, output:
> 473 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/passwd'
> 474 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/shadow'
> 475 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/group'
> 476 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/gshadow'
> 477 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 478 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
> 479 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
> 480 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 481 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 482 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 483 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 484
> 485 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> SSH command on esx35-hypervisor-1:
> 486 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> 487
> 488 |12719|9:9|deleted| ---- WARNING ----
> 489 |12719|9:9|deleted| 2010-04-01 15:38:18|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 1/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> admin, exit status: 255, SSH exits with the exit status of the remote
> command or with 255 if an error occurred, output:
> 490 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/passwd'
> 491 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/shadow'
> 492 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/group'
> 493 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/gshadow'
> 494 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 495 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 496 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> 497 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 498 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 499 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 500 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 501
> 502 2010-04-01 15:38:18|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 2 seconds before making next SSH attempt
> 503 2010-04-01 15:38:19|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:38:19
> 504 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 2/3: executing SSH command on esx35-hypervisor-1:
> 505 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> 506
> 507 |12719|9:9|deleted| ---- WARNING ----
> 508 |12719|9:9|deleted| 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 2/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> admin, exit status: 255, SSH exits with the exit status of the remote
> command or with 255 if an error occurred, output:
> 509 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/passwd'
> 510 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/shadow'
> 511 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/group'
> 512 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/gshadow'
> 513 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 514 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 515 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> 516 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 517 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 518 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 519 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 520
> 521 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 15 seconds before making next SSH attempt
> 522 2010-04-01 15:38:25|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:38:24
> 523 2010-04-01 15:38:30|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:38:30
> 524 2010-04-01 15:38:35|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:38:35
> 525 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 3/3: executing SSH command on esx35-hypervisor-1:
> 526 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
> 527
> 528 |12719|9:9|deleted| ---- WARNING ----
> 529 |12719|9:9|deleted| 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 3/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel
> admin, exit status: 255, SSH exits with the exit status of the remote
> command or with 255 if an error occurred, output:
> 530 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/passwd'
> 531 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/shadow'
> 532 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/group'
> 533 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/gshadow'
> 534 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 535 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 536 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> 537 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 538 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 539 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 540 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 541
> 542
> 543 |12719|9:9|deleted| ---- WARNING ----
> 544 |12719|9:9|deleted| 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel
> admin' 2>&1, exit status: 255, output:
> 545 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/passwd'
> 546 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/shadow'
> 547 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/group'
> 548 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find
> 'admin' in '/etc/gshadow'
> 549 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 550 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
> 551 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
> 552 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 553 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 554 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 555 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 556
> 557 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> SSH command on esx35-hypervisor-1:
> 558 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> /etc/ssh/external_sshd_config' 2>&1
> 559
> 560 |12719|9:9|deleted| ---- WARNING ----
> 561 |12719|9:9|deleted| 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
> exits with the exit status of the remote command or with 255 if an error
> occurred, output:
> 562 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> /etc/ssh/external_sshd_config: No such file or directory
> 563 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 564 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 565 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> 566 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 567 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 568 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 569 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 570
> 571 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 2 seconds before making next SSH attempt
> 572 2010-04-01 15:38:37|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 2/3: executing SSH command on esx35-hypervisor-1:
> 573 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> /etc/ssh/external_sshd_config' 2>&1
> 574
> 575 |12719|9:9|deleted| ---- WARNING ----
> 576 |12719|9:9|deleted| 2010-04-01 15:38:38|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
> exits with the exit status of the remote command or with 255 if an error
> occurred, output:
> 577 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> /etc/ssh/external_sshd_config: No such file or directory
> 578 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 579 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 580 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> 581 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 582 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 583 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 584 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 585
> 586 2010-04-01 15:38:38|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 15 seconds before making next SSH attempt
> 587 2010-04-01 15:38:40|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:38:40
> 588 2010-04-01 15:38:45|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:38:45
> 589 2010-04-01 15:38:50|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:38:50
> 590 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 3/3: executing SSH command on esx35-hypervisor-1:
> 591 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d"
> /etc/ssh/external_sshd_config' 2>&1
> 592
> 593 |12719|9:9|deleted| ---- WARNING ----
> 594 |12719|9:9|deleted| 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie
> "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH
> exits with the exit status of the remote command or with 255 if an error
> occurred, output:
> 595 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> /etc/ssh/external_sshd_config: No such file or directory
> 596 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 597 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 598 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> 599 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 600 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 601 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 602 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 603
> 604
> 605 |12719|9:9|deleted| ---- WARNING ----
> 606 |12719|9:9|deleted| 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^AllowUsers
> .*/d" /etc/ssh/external_sshd_config' 2>&1, exit status: 255, output:
> 607 |12719|9:9|deleted| ssh output (sed -ie "/...): sed:
> /etc/ssh/external_sshd_config: No such file or directory
> 608 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 609 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
> 610 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
> 611 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 612 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 613 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 614 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 615
> 616 No recipient addresses found in header
> 617 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:mail(1348)|SUCCESS
> -- Sending mail To: , PROBLEM -- Linux.pm
> 618
> 619 |12719|9:9|deleted| ---- CRITICAL ----
> 620 |12719|9:9|deleted| 2010-04-01
> 15:38:53|12719|9:9|deleted|Linux.pm:delete_user(566)|failed to add
> AllowUsers admin to external_sshd_config
> 621 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 622 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 566)
> 623 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
> 624 |12719|9:9|deleted| (-3) reclaim.pm, process (line: 173)
> 625 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
> 626 |12719|9:9|deleted| (-5) vcld, main (line: 341)
> 627
> 628 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing
> SSH command on esx35-hypervisor-1:
> 629 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> 630
> 631 |12719|9:9|deleted| ---- WARNING ----
> 632 |12719|9:9|deleted| 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
> .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
> remote command or with 255 if an error occurred, output:
> 633 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> No such file or directory
> 634 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 635 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 636 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> 637 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 638 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 639 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 640 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 641
> 642 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 2 seconds before making next SSH attempt
> 643 2010-04-01 15:38:55|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:38:55
> 644 2010-04-01 15:38:55|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 2/3: executing SSH command on esx35-hypervisor-1:
> 645 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> 646
> 647 |12719|9:9|deleted| ---- WARNING ----
> 648 |12719|9:9|deleted| 2010-04-01 15:38:56|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
> .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
> remote command or with 255 if an error occurred, output:
> 649 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> No such file or directory
> 650 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 651 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 652 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> 653 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 654 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 655 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 656 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 657
> 658 2010-04-01 15:38:56|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping
> for 15 seconds before making next SSH attempt
> 659 2010-04-01 15:39:00|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:00
> 660 2010-04-01 15:39:05|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:05
> 661 2010-04-01 15:39:10|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:10
> 662 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt
> 3/3: executing SSH command on esx35-hypervisor-1:
> 663 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22
> -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
> 664
> 665 |12719|9:9|deleted| ---- WARNING ----
> 666 |12719|9:9|deleted| 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt
> 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin
> .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the
> remote command or with 255 if an error occurred, output:
> 667 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> No such file or directory
> 668 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 669 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
> 670 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> 671 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 672 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 673 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 674 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 675
> 676
> 677 |12719|9:9|deleted| ---- WARNING ----
> 678 |12719|9:9|deleted| 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed
> to run SSH command after 3 attempts, command: /usr/bin/ssh -i
> /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^admin
> .*/d" /etc/sudoers' 2>&1, exit status: 255, output:
> 679 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers:
> No such file or directory
> 680 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 681 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
> 682 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
> 683 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
> 684 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
> 685 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
> 686 |12719|9:9|deleted| (-6) vcld, main (line: 341)
> 687
> 688 No recipient addresses found in header
> 689 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:mail(1348)|SUCCESS
> -- Sending mail To: , PROBLEM -- Linux.pm
> 690
> 691 |12719|9:9|deleted| ---- CRITICAL ----
> 692 |12719|9:9|deleted| 2010-04-01
> 15:39:11|12719|9:9|deleted|Linux.pm:delete_user(578)|failed to clear admin
> from /etc/sudoers
> 693 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
> 694 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 578)
> 695 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
> 696 |12719|9:9|deleted| (-3) reclaim.pm, process (line: 173)
> 697 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
> 698 |12719|9:9|deleted| (-5) vcld, main (line: 341)
> 699
> 700 2010-04-01 15:39:11|12719|9:9|deleted|Linux.pm:sanitize(852)|users
> have been deleted from esx35-hypervisor-1
> 701 2010-04-01 15:39:11|12719|9:9|deleted|reclaim.pm:process(173)|esx35-hypervisor-1
> has been sanitized
> 702 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8018)|called
> from VCL::reclaim::process(271)
> 703 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8054)|parent:
> parent reservation ID for this request: 9
> 704 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> entry not inserted, reservation has been deleted
> 705 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:update_request_state(2186)|request
> 9 state updated to: complete, laststate to: deleted
> 706 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8078)|req=9:
> request state changed: deleted->complete, laststate: reserved->deleted
> 707 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> entry not inserted, reservation has been deleted
> 708 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> entry not inserted, reservation has been deleted
> 709 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:update_computer_state(2228)|computer
> 2 state updated to: available
> 710 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8096)|req=9:
> computer esx35-hypervisor-1 state changed: available->available
> 711 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8106)|req=9:
> log table id=10 will not be updated
> 712 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog
> entry not inserted, reservation has been deleted
> 713 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8122)|req=9:
> process exiting
> 714 2010-04-01
> 15:39:11|12719|9:9|deleted|State.pm:DESTROY(905)|destructor called,
> ref($self)=VCL::reclaim
> 715 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:delete_computerloadlog_reservation(7551)|removing
> computerloadlog entries matching loadstate = begin
> 716 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:delete_computerloadlog_reservation(7598)|deleted
> rows from computerloadlog for reservation id=9
> 717 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(912)|removed
> computerloadlog rows with loadstate=begin for reservation
> 718 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(924)|number
> of database handles state process created: 1
> 719 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(933)|process
> has a database handle stored in $ENV{dbh}, attempting disconnect
> 720 2010-04-01
> 15:39:11|12719|9:9|deleted|State.pm:DESTROY(935)|$ENV{dbh}: database
> disconnect successful
> 721 2010-04-01
> 15:39:11|12719|9:9|deleted|State.pm:DESTROY(949)|VCL::reclaim process 12719
> exiting
> 722 2010-04-01 15:39:11|5754|vcld:REAPER(744)|VCL process exited for
> reservation 9
> 723 2010-04-01 15:39:11|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:11
> 724 2010-04-01 15:39:11|5754|9:9|complete|vcld:main(250)|request deleted
> 725 2010-04-01 15:39:16|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:16
> 726 2010-04-01 15:39:21|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:21
> 727 2010-04-01 15:39:26|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:26
> 728 2010-04-01 15:39:31|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:31
> 729 2010-04-01 15:39:36|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:36
> 730 2010-04-01 15:39:42|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:42
> 731 2010-04-01 15:39:47|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:47
> 732 2010-04-01 15:39:52|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:52
> 733 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> output:
> 734 |12726|10:10|new| none
> 735 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> command executed on 192.168.129.37, returning (0, "none")
> 736 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing
> SSH command on 192.168.129.37:
> 737 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> 192.168.129.37 'grep adapterType
> /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk
> 2>&1' 2>&1
> 738 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command
> output:
> 739 |12726|10:10|new| ddb.adapterType = "buslogic"
> 740 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH
> command executed on 192.168.129.37, returning (0, "ddb.adapterType =
> "buslogic"")
> 741 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(311)|adapter=
> buslogic
> 742 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(342)|eth0 MAC
> address set for vcl assigned
> 743 Use of uninitialized value in concatenation (.) or string at
> 744 /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pmline 343 (#1)
> 745 (W uninitialized) An undefined value was used as if it were
> already
> 746 defined. It was interpreted as a "" or a 0, but maybe it was a
> mistake.
> 747 To suppress this warning assign a defined value to your
> variables.
> 748
> 749 To help you figure out what was undefined, perl tells you what
> operation
> 750 you used the undefined value in. Note, however, that perl
> optimizes your
> 751 program and the operation displayed in the warning may not
> necessarily
> 752 appear literally in your program. For example, "that $foo" is
> 753 usually optimized into "that " . $foo, and the warning will
> refer to
> 754 the concatenation (.) operator, even though there is no . in
> your
> 755 program.
> 756
> 757
> 758 |12726|10:10|new| ---- WARNING ----
> 759 |12726|10:10|new| 2010-04-01
> 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> value in concatenation (.) or string at
> /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 343.
> 760 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 761 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> 762 |12726|10:10|new| (-2) esx.pm, load (line: 343)
> 763 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
> 764 |12726|10:10|new| (-4) new.pm, process (line: 266)
> 765 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> 766 |12726|10:10|new| (-6) vcld, main (line: 341)
> 767
> 768
> 769 |12726|10:10|new| ---- WARNING ----
> 770 |12726|10:10|new| 2010-04-01
> 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> value in concatenation (.) or string at
> /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 353.
> 771 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 772 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> 773 |12726|10:10|new| (-2) esx.pm, load (line: 353)
> 774 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
> 775 |12726|10:10|new| (-4) new.pm, process (line: 266)
> 776 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> 777 |12726|10:10|new| (-6) vcld, main (line: 341)
> 778
> 779 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(353)|eth1 MAC
> address set for vcl assigned
> 780
> 781 |12726|10:10|new| ---- WARNING ----
> 782 |12726|10:10|new| 2010-04-01
> 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized
> value in concatenation (.) or string at
> /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 354.
> 783 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 784 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
> 785 |12726|10:10|new| (-2) esx.pm, load (line: 354)
> 786 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
> 787 |12726|10:10|new| (-4) new.pm, process (line: 266)
> 788 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
> 789 |12726|10:10|new| (-6) vcld, main (line: 341)
> 790
> 791 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(376)|wrote vmxarray
> to /tmp/esx3-basewinxp-v0.vmx
> 792 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_scp_command(6403)|attempt
> 1/3: executing SCP command: /usr/bin/scp -B -i /etc/vcl/vcl.key -P 22 -p -r
> /tmp/esx3-basewinxp-v0.vmx 192.168.129.37:/install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmx
> 2>&1
> 793 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_scp_command(6453)|scp
> successful: attempt 1/3, exit status: 0, output: none
> 794 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(401)|Register
> Command: /usr/lib/vmware-vcli/apps/vm/vmregister.pl --server
> 'esx35-hypervisor-1' --username root --password 'xxxxxxxxxxx' --vmxpath
> '[VCL]/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmx' --operation register
> --vmname esx3-basewinxp-v0 --pool Resources --hostname '
> esx35-hypervisor-1.odu.edu' --datacenter 'ha-datacenter'
> 795 2010-04-01 15:39:57|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:39:57
> 796 2010-04-01 15:39:57|12726|10:10|new|esx.pm:load(404)|Registered:
> 797 |12726|10:10|new| SOAP Fault:
> 798 |12726|10:10|new| -----------
> 799 |12726|10:10|new| Fault string: fault.RestrictedVersion.summary
> 800 |12726|10:10|new| Fault detail: RestrictedVersionFault
> 801 2010-04-01 15:39:57|12726|10:10|new|esx.pm:load(413)|Power on
> command: /usr/lib/vmware-vcli/apps/vm/vmcontrol.pl --server
> 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --operation poweron
> --username root --password 'xxxxxxxxxxx'
> 802 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(416)|Powered on:
> Virtual Machine esx3-basewinxp-v0 not found.
> 803 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(505)|IP is known
> for esx3-basewinxp-v0
> 804 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(511)|Waiting for
> ssh to come up on esx3-basewinxp-v0
> 805 2010-04-01 15:40:00|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 1
> 806 2010-04-01 15:40:02|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:02
> 807 2010-04-01 15:40:06|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 2
> 808 2010-04-01 15:40:07|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:07
> 809 2010-04-01 15:40:12|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:12
> 810 2010-04-01 15:40:12|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 3
> 811 2010-04-01 15:40:17|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:17
> 812 2010-04-01 15:40:18|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 4
> 813 2010-04-01 15:40:22|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:22
> 814 2010-04-01 15:40:24|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 5
> 815 2010-04-01 15:40:27|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:27
> 816 2010-04-01 15:40:30|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 6
> 817 2010-04-01 15:40:32|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:32
> 818 2010-04-01 15:40:36|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 7
> 819 2010-04-01 15:40:37|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:37
> 820 2010-04-01 15:40:42|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:42
> 821 2010-04-01 15:40:42|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 8
> 822 2010-04-01 15:40:47|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:47
> 823 2010-04-01 15:40:48|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 9
> 824 2010-04-01 15:40:52|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:52
> 825 2010-04-01 15:40:54|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 10
> 826 2010-04-01 15:40:57|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:40:57
> 827 2010-04-01 15:41:00|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 11
> 828 2010-04-01 15:41:02|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:02
> 829 2010-04-01 15:41:06|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 12
> 830 2010-04-01 15:41:07|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:07
> 831 2010-04-01 15:41:12|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:12
> 832 2010-04-01 15:41:12|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 13
> 833 2010-04-01 15:41:17|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:17
> 834 2010-04-01 15:41:18|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 14
> 835 2010-04-01 15:41:22|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:22
> 836 2010-04-01 15:41:24|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 15
> 837 2010-04-01 15:41:27|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:27
> 838 2010-04-01 15:41:30|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 16
> 839 2010-04-01 15:41:32|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:32
> 840 2010-04-01 15:41:36|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 17
> 841 2010-04-01 15:41:37|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:37
> 842 2010-04-01 15:41:42|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:42
> 843 2010-04-01 15:41:42|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 18
> 844 2010-04-01 15:41:47|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:47
> 845 2010-04-01 15:41:48|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 19
> 846 2010-04-01 15:41:52|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:52
> 847 2010-04-01 15:41:54|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 20
> 848 2010-04-01 15:41:57|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:41:57
> 849 2010-04-01 15:42:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 21
> 850 2010-04-01 15:42:02|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:02
> 851 2010-04-01 15:42:07|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 22
> 852 2010-04-01 15:42:07|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:07
> 853 2010-04-01 15:42:12|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:12
> 854 2010-04-01 15:42:13|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 23
> 855 2010-04-01 15:42:18|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:18
> 856 2010-04-01 15:42:19|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 24
> 857 2010-04-01 15:42:23|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:23
> 858 2010-04-01 15:42:25|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 25
> 859 2010-04-01 15:42:28|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:28
> 860 2010-04-01 15:42:31|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 26
> 861 2010-04-01 15:42:33|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:33
> 862 2010-04-01 15:42:37|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 27
> 863 2010-04-01 15:42:38|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:38
> 864 2010-04-01 15:42:43|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 28
> 865 2010-04-01 15:42:43|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:43
> 866 2010-04-01 15:42:48|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:48
> 867 2010-04-01 15:42:49|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 29
> 868 2010-04-01 15:42:53|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:53
> 869 2010-04-01 15:42:55|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 30
> 870 2010-04-01 15:42:58|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:42:58
> 871 2010-04-01 15:43:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 31
> 872 2010-04-01 15:43:03|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:03
> 873 2010-04-01 15:43:07|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 32
> 874 2010-04-01 15:43:08|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:08
> 875 2010-04-01 15:43:13|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 33
> 876 2010-04-01 15:43:13|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:13
> 877 2010-04-01 15:43:18|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:18
> 878 2010-04-01 15:43:19|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 34
> 879 2010-04-01 15:43:23|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:23
> 880 2010-04-01 15:43:25|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 35
> 881 2010-04-01 15:43:28|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:28
> 882 2010-04-01 15:43:31|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 36
> 883 2010-04-01 15:43:33|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:33
> 884 2010-04-01 15:43:37|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 37
> 885 2010-04-01 15:43:38|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:38
> 886 2010-04-01 15:43:43|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 38
> 887 2010-04-01 15:43:43|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:43
> 888 2010-04-01 15:43:48|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:48
> 889 2010-04-01 15:43:49|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 39
> 890 2010-04-01 15:43:53|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:53
> 891 2010-04-01 15:43:55|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 40
> 892 2010-04-01 15:43:58|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:43:58
> 893 2010-04-01 15:44:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 41
> 894 2010-04-01 15:44:03|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:03
> 895 2010-04-01 15:44:07|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 42
> 896 2010-04-01 15:44:08|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:08
> 897 2010-04-01 15:44:13|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 43
> 898 2010-04-01 15:44:13|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:13
> 899 2010-04-01 15:44:18|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:18
> 900 2010-04-01 15:44:19|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 44
> 901 2010-04-01 15:44:23|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:23
> 902 2010-04-01 15:44:25|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 45
> 903 2010-04-01 15:44:29|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:28
> 904 2010-04-01 15:44:31|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 46
> 905 2010-04-01 15:44:34|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:34
> 906 2010-04-01 15:44:37|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 47
> 907 2010-04-01 15:44:39|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:39
> 908 2010-04-01 15:44:43|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 48
> 909 2010-04-01 15:44:44|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:44
> 910 2010-04-01 15:44:49|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:49
> 911 2010-04-01 15:44:49|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 49
> 912 2010-04-01 15:44:54|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:54
> 913 2010-04-01 15:44:55|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 50
> 914 2010-04-01 15:44:59|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:44:59
> 915 2010-04-01 15:45:01|12726|10:10|new|esx.pm:load(528)|going to sleep
> 5 seconds, waiting for computer to start SSH. Try 51
> 916 2010-04-01 15:45:04|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:45:04
> 917 No recipient addresses found in header
> 918 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
> Sending mail To: , PROBLEM -- esx.pm
> 919
> 920 |12726|10:10|new| ---- CRITICAL ----
> 921 |12726|10:10|new| 2010-04-01 15:45:07|12726|10:10|new|esx.pm:load(521)|waited
> acceptable amount of time for sshd to become active, please check
> esx3-basewinxp-v0 on esx35-hypervisor-1
> 922 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 923 |12726|10:10|new| (-1) esx.pm, load (line: 521)
> 924 |12726|10:10|new| (-2) new.pm, reload_image (line: 665)
> 925 |12726|10:10|new| (-3) new.pm, process (line: 266)
> 926 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
> 927 |12726|10:10|new| (-5) vcld, main (line: 341)
> 928
> 929
> 930 |12726|10:10|new| ---- WARNING ----
> 931 |12726|10:10|new| 2010-04-01 15:45:08|12726|10:10|new|new.pm:reload_image(670)|esx3-basewinxp-v0
> failed to load on esx3-basewinxp-v0, returning
> 932 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 933 |12726|10:10|new| (-1) new.pm, reload_image (line: 670)
> 934 |12726|10:10|new| (-2) new.pm, process (line: 266)
> 935 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
> 936 |12726|10:10|new| (-4) vcld, main (line: 341)
> 937
> 938 2010-04-01 15:45:08|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> computer=3, loadimagefailed, esx3-basewinxp-v0 failed to load on
> esx3-basewinxp-v0
> 939
> 940 |12726|10:10|new| ---- WARNING ----
> 941 |12726|10:10|new| 2010-04-01 15:45:08|12726|10:10|new|new.pm:process(313)|failed
> to load esx3-basewinxp-v0 with esx3-basewinxp-v0
> 942 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 943 |12726|10:10|new| (-1) new.pm, process (line: 313)
> 944 |12726|10:10|new| (-2) vcld, make_new_child (line: 594)
> 945 |12726|10:10|new| (-3) vcld, main (line: 341)
> 946
> 947 2010-04-01
> 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> to retrieve current state of computer esx3-basewinxp-v0 from the database
> 948 2010-04-01
> 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> current state of computer esx3-basewinxp-v0 from the database: reloading
> 949 2010-04-01
> 15:45:08|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure
> updated: $self->request_data->{reservation}{10}{computer}{state}{name}
> 950 |12726|10:10|new| computer_state_name = reloading
> 951 No recipient addresses found in header
> 952 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS --
> Sending mail To: , PROBLEM -- State.pm
> 953
> 954 |12726|10:10|new| ---- CRITICAL ----
> 955 |12726|10:10|new| 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:reservation_failed(290)|reservation failed
> on esx3-basewinxp-v0: process failed after trying to load or make available
> 956 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
> 957 |12726|10:10|new| (-1) State.pm, reservation_failed (line: 290)
> 958 |12726|10:10|new| (-2) new.pm, process (line: 316)
> 959 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
> 960 |12726|10:10|new| (-4) vcld, main (line: 341)
> 961
> 962 2010-04-01 15:45:08|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted
> computer=3, failed, process failed after trying to load or make available
> 963 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:reservation_failed(293)|inserted
> computerloadlog entry
> 964 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:reservation_failed(301)|updated log ending
> value to 'failed', logid=11
> 965 2010-04-01 15:45:08|12726|10:10|new|utils.pm:update_computer_state(2228)|computer
> 3 state updated to: failed
> 966 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:reservation_failed(312)|computer
> esx3-basewinxp-v0 (3) state set to failed
> 967 2010-04-01 15:45:08|12726|10:10|new|utils.pm:update_request_state(2186)|request
> 10 state updated to: failed, laststate to: new
> 968 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:reservation_failed(325)|set request state
> to 'failed'/'new'
> 969 2010-04-01 15:45:08|12726|10:10|new|utils.pm:is_inblockrequest(6972)|zero
> rows were returned from database select
> 970 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:reservation_failed(343)|esx3-basewinxp-v0
> is NOT in blockcomputers table
> 971 2010-04-01
> 15:45:08|12726|10:10|new|State.pm:reservation_failed(346)|exiting 1
> 972 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(905)|destructor
> called, ref($self)=VCL::new
> 973 2010-04-01 15:45:08|12726|10:10|new|utils.pm:delete_computerloadlog_reservation(7551)|removing
> computerloadlog entries matching loadstate = begin
> 974 2010-04-01 15:45:08|12726|10:10|new|utils.pm:delete_computerloadlog_reservation(7598)|deleted
> rows from computerloadlog for reservation id=10
> 975 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(912)|removed
> computerloadlog rows with loadstate=begin for reservation
> 976 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(924)|number of
> database handles state process created: 1
> 977 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(933)|process
> has a database handle stored in $ENV{dbh}, attempting disconnect
> 978 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(935)|$ENV{dbh}:
> database disconnect successful
> 979 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(949)|VCL::new
> process 12726 exiting
> 980 2010-04-01 15:45:08|5754|vcld:REAPER(744)|VCL process exited for
> reservation 10
> 981 2010-04-01 15:45:08|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:45:08
> 982 2010-04-01 15:45:13|5754|vcld:main(165)|lastcheckin time updated for
> management node 1: 2010-04-01 15:45:13
>
> =====================================================================================
>
> Thank you,
> Amit
>
>
>
RE: [Spam: 6.0] Re: VCL Getting Reservation to work
Posted by "Kumar, Amit H." <AH...@odu.edu>.
The Savior! Thank you Ryan getting back!!!
Not sure if this is the best format for readability...please let me know otherwise.
I am also a recent log that I tried with verbose ssh enabled etc, I haven't looked at it yet. It went past my previous log. .....................
Image: Table: Rows: 10
====================
id name prettyname ownerid platformid OSid imagemetaid minram minprocnumber minprocspeed minnetwork maxconcurrent reloadtime deleted test lastupdate forcheckout maxinitialtime project size architecture description usage basedoffrevisionid
1 winxp-base1-v0 No Apps (WinXP) 1 [->admin] 1 [->i386] 7 [->winxp] NULL 0 1 0 10 NULL 14 0 0 2007-04-11 16:07:38 1 0 vcl 1045 x86 NULL NULL 0
2 rhel4-base2-v0 Red Hat Enterprise Linux 4.4 Base (KS) 1 [->admin] 1 [->i386] 12 [->rhel4] NULL 1024 1 1024 100 NULL 13 0 0 2007-03-02 16:33:33 1 0 vcl 0 x86 NULL NULL 0
3 rh4image-base3-v0 RHEL4 base (image) 1 [->admin] 1 [->i386] 13 [->rh4image] NULL 10 1 1024 100 NULL 10 0 0 2007-01-24 15:02:07 1 0 vcl 1450 x86 NULL NULL 0
4 noimage No Image 1 [->admin] 1 [->i386] 2 [->sun4x_58] NULL 0 1 0 10 NULL 0 0 0 NULL 1 0 vcl 1450 x86 NULL NULL 0
5 rhfc5-fc5base5-v0 Red Hat Fedora Core 5 base (KS) 1 [->admin] 1 [->i386] 15 [->rhfc5] NULL 1024 1 1024 100 NULL 13 0 0 2006-10-02 10:04:24 1 0 vcl 1450 x86 NULL NULL 0
6 fc5image-FC5baseRHFC56-v0 FC5base(RHFC5) 1 [->admin] 1 [->i386] 14 [->fc5image] NULL 64 1 500 10 NULL 10 0 0 2006-10-02 14:13:26 1 0 vcl 0 x86 NULL NULL 0
7 vmwarewinxp-base7-v1 No Apps (WinXP vmware) 1 [->admin] 1 [->i386] 16 [->vmwarewinxp] NULL 512 1 1024 100 NULL 5 0 0 2007-04-04 09:45:38 1 0 vcl 3244 x86 NULL NULL 0
8 rh4image-VMwareserverhostRHEL48-v3 VMware server host (RHEL4) 1 [->admin] 1 [->i386] 13 [->rh4image] NULL 64 1 500 10 NULL 10 0 0 2007-11-26 11:23:53 1 0 vcl 1450 x86 NULL NULL 0
9 esx35-base-v0 VMware ESX 3.5 standard server 1 [->admin] 1 [->i386] 20 [->esx35] NULL 2048 2 2000 100 NULL 9 0 0 2008-03-24 14:23:54 1 0 vcl 1450 x86 NULL NULL 0
10 esx3-basewinxp-v0 No Apps (ESXi WinXP) 1 [->admin] 1 [->i386] 20 [->esx35] NULL 512 1 1024 100 NULL 9 0 0 2010-03-24 17:35:12 1 0 vcl 4100 x86 NULL NULL 0
/End Image Table====================
ImageRevision Table: Rows: 11
===========================
id imageid revision userid datecreated deleted datedeleted production comments imagename
1 1 [->No Apps (WinXP)] 0 1 [->admin] 2007-03-01 14:46:26 0 NULL 1 NULL winxp-base1-v0
2 2 [->Red Hat Enterprise Linux 4.4 Base (KS)] 0 1 [->admin] 2006-09-22 16:33:24 0 NULL 1 NULL rhel4-base2-v0
3 3 [->RHEL4 base (image)] 0 1 [->admin] 2007-01-24 15:04:11 0 NULL 1 NULL rh4image-base3-v0
4 4 [->No Image] 0 1 [->admin] 1980-01-01 00:00:00 0 NULL 1 NULL noimage
5 5 [->Red Hat Fedora Core 5 base (KS)] 0 1 [->admin] 2006-10-02 10:04:24 0 NULL 1 NULL rhfc5-fc5base5-v0
6 6 [->FC5base(RHFC5)] 0 1 [->admin] 2006-10-02 14:13:26 0 NULL 1 NULL fc5image-FC5baseRHFC56-v0
7 7 [->No Apps (WinXP vmware)] 0 1 [->admin] 2007-04-03 11:24:12 0 NULL 1 NULL vmwarewinxp-base7-v0
8 8 [->VMware server host (RHEL4)] 0 1 [->admin] 2007-01-30 10:50:56 0 NULL 1 NULL rh4image-VMwareserverhostRHEL48-v0
9 9 [->VMware ESX 3.5 standard server] 0 1 [->admin] 2008-03-24 14:23:54 0 NULL 1 NULL esx35-base-v0
10 7 [->No Apps (WinXP vmware)] 1 1 [->admin] 2010-03-18 14:56:50 0 NULL 1 NULL vmwarewinxp-base7-v1
11 10 [->No Apps (ESXi WinXP)] 0 1 [->admin] 2010-03-24 17:35:12 0 NULL 1 NULL esx3-basewinxp-v0
/End ImageRevision Table====================
Computer Table: Rows: 3
============================================
id stateid ownerid platformid scheduleid currentimageid preferredimageid nextimageid imagerevisionid RAM procnumber procspeed network hostname IPaddress privateIPaddress eth0macaddress eth1macaddress type provisioningid drivetype deleted notes lastcheck location dsa dsapub rsa rsapub host hostpub vmhostid vmtypeid
1 10 [->maintenance] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0 0 512 2 2200 1000 resxi-2.odu.edu 128.82.128.218 NULL NULL NULL virtualmachine 4 [->vmware_esx] hda 1 NULL NULL NULL NULL NULL NULL NULL [BLOB - 0B] NULL NULL NULL
2 2 [->available] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0 0 20480 8 2200 1000 esx35-hypervisor-1.odu.edu 128.82.128.252 NULL NULL NULL blade 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL [BLOB - 0B] NULL NULL NULL
3 5 [->failed] 1 [->admin] 1 [->i386] 1 [->VCL 24x7] 4 [->No Image] 0 0 0 512 2 2200 100 esx3-basewinxp-v0.odu.edu 128.82.128.217 NULL NULL NULL virtualmachine 4 [->vmware_esx] hda 0 NULL NULL NULL NULL NULL NULL [BLOB - 0B] NULL 1 NULL
/End Computer Table======================================
Vmprofile Table: Rows: 4
========================
id profilename vmtypeid imageid nasshare datastorepath vmpath virtualswitch0 virtualswitch1 vmdisk username password
1 VMware GSX standard 3 [->vmwareGSX] 8 [->VMware server host (RHEL4)] NULL /var/lib/vmware/Virtual Machines NULL VMnet0 VMnet2 localdisk NULL NULL
2 Vmware ESX standard network mounted share 5 [->vmwareESX3] 9 [->VMware ESX 3.5 standard server] NULL /vmfs/volumes/nfs1 /vmfs/volumes/storage1 VM Network Virtual Machine Public Network networkdisk NULL NULL
3 Vmware ESX standard localdisk 6 [->vmwareESXi] 9 [->VMware ESX 3.5 standard server] NULL 192.168.129.37:/install/image/vmware_images 192.168.129.37:/install/image/vmware_images VM Network Private Vlan328 localdisk root
4 Vmware ESX SAN 5 [->vmwareESX3] 9 [->VMware ESX 3.5 standard server] NULL /vmfs/volumes/NetApp /vmfs/volumes/storage1 Intranet2 MCNC Public networkdisk NULL NULL
/End Vmprofile===============================
VMhost Table: Rows: 1
=====================
id computerid vmlimit vmprofileid vmkernalnic vmwaredisk
1 2 [->esx35-hypervisor-1.odu.edu] 2 3 NULL localdisk
/End Vmhost Table==========================
================================= LOG FILE ===========================================
13 2010-04-01 15:37:19|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:37:19
14 2010-04-01 15:37:24|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:37:24
15 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:reservation_being_processed(9634)|computerloadlog 'begin' entry does NOT exist for reservation 9
16 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:reservation_being_processed(9686)|reservation is NOT currently being processed
17 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(276)|reservation 9 is NOT already being processed
18
19 |5754|9:9|deleted| ---- WARNING ----
20 |5754|9:9|deleted| 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:get_request_info(5218)|preferredimageid is not set for computer id=2
21 |5754|9:9|deleted| ( 0) utils.pm, notify (line: 737)
22 |5754|9:9|deleted| (-1) utils.pm, get_request_info (line: 5218)
23 |5754|9:9|deleted| (-2) vcld, main (line: 280)
24
25 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:get_request_info(5354)|standalone affiliation found: Local
26 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:get_management_node_info(6815)|management node info retrieved from database for colbert
27 2010-04-01 15:37:24|5754|9:9|deleted|vcld:main(281)|retrieved request information from database
28 2010-04-01 15:37:24|5754|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 9
29 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:update_request_state(2186)|request 9 state updated to: pending, laststate to: deleted
30 2010-04-01 15:37:24|5754|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog entry not inserted, reservation has been deleted
31 2010-04-01 15:37:24|5754|9:9|deleted|vcld:make_new_child(489)|request will be processed by reclaim.pm
32 2010-04-01 15:37:24|5754|9:9|deleted|vcld:make_new_child(509)|loaded VCL::reclaim module
33 2010-04-01 15:37:24|12719|9:9|deleted|vcld:make_new_child(582)|vcld environment variable set to 0 for this process
34 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor called, class=VCL::reclaim
35 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::reclaim object created
36 2010-04-01 15:37:24|5754|9:9|deleted|vcld:make_new_child(566)|current number of forked kids: 1
37 2010-04-01 15:37:24|12719|9:9|deleted|State.pm:initialize(85)|obtained a database handle for this state process, stored as $ENV{dbh}
38 2010-04-01 15:37:24|12719|9:9|deleted|State.pm:check_image_os(820)|no corrections need to be made, not an imaging request, returning 1
39 2010-04-01 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 9
40 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7863)|reservation count: 1
41 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7882)|PARENTIMAGE: 1
42 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7883)|SUBIMAGE: 0
43 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:rename_vcld_process(7901)|renamed process to 'vcld VCL::reclaim 9:9 deleted'
44 2010-04-01 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 9
45 2010-04-01 15:37:24|12719|9:9|deleted|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 9
46 2010-04-01 15:37:24|12719|9:9|deleted|State.pm:initialize(121)|attempting to load provisioning module: VCL::Module::Provisioning::esx
47 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(138)|esx vmware toolkit root path found: /usr/lib/vmware-vcli/apps
48 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(140)|vmware ESX module initialized
49 2010-04-01 15:37:24|12719|9:9|deleted|State.pm:initialize(127)|VCL::Module::Provisioning::esx module loaded
50 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor called, class=VCL::Module::Provisioning::esx
51 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::Provisioning::esx object created
52 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(138)|esx vmware toolkit root path found: /usr/lib/vmware-vcli/apps
53 2010-04-01 15:37:24|12719|9:9|deleted|esx.pm:initialize(140)|vmware ESX module initialized
54 2010-04-01 15:37:24|12719|9:9|deleted|State.pm:initialize(131)|VCL::Module::Provisioning::esx provisioner object created
55 2010-04-01 15:37:24|12719|9:9|deleted|State.pm:initialize(145)|attempting to load OS module: VCL::Module::OS::Linux
56 2010-04-01 15:37:24|12719|9:9|deleted|State.pm:initialize(151)|VCL::Module::OS::Linux module loaded
57 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(132)|constructor called, class=VCL::Module::OS::Linux
58 2010-04-01 15:37:24|12719|9:9|deleted|Module.pm:new(154)|VCL::Module::OS::Linux object created
59 2010-04-01 15:37:24|12719|9:9|deleted|State.pm:initialize(154)|VCL::Module::OS::Linux OS object created
60 2010-04-01 15:37:24|12719|9:9|deleted|State.pm:initialize(169)|returning 1
61 2010-04-01 15:37:24|12719|9:9|deleted|vcld:make_new_child(591)|VCL::reclaim object created and initialized
62 2010-04-01 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation remote IP is not defined
63 2010-04-01 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1946)|attempting to retrieve current state of computer esx35-hypervisor-1 from the database
64 2010-04-01 15:37:24|12719|9:9|deleted|DataStructure.pm:get_computer_state_name(1977)|retrieved current state of computer esx35-hypervisor-1 from the database: available
65 2010-04-01 15:37:24|12719|9:9|deleted|DataStructure.pm:_automethod(697)|data structure updated: $self->request_data->{reservation}{9}{computer}{state}{name}
66 |12719|9:9|deleted| computer_state_name = available
67 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog entry not inserted, reservation has been deleted
68 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog entry not inserted, reservation has been deleted
69 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog entry not inserted, reservation has been deleted
70 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog entry not inserted, reservation has been deleted
71 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog entry not inserted, reservation has been deleted
72 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(138)|computer type is blade
73 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(142)|request laststate is reserved, attempting to sanitize computer
74 2010-04-01 15:37:24|12719|9:9|deleted|reclaim.pm:process(172)|calling VCL::Module::OS::Linux::sanitize() subroutine
75 2010-04-01 15:37:24|12719|9:9|deleted|DataStructure.pm:get_reservation_remote_ip(956)|reservation remote IP is not defined
76 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing SSH command on esx35-hypervisor-1:
77 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'netstat -an' 2>&1
78
79 |12719|9:9|deleted| ---- WARNING ----
80 |12719|9:9|deleted| 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 1/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
81 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not found
82 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
83 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
84 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
85 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
86 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
87 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
88 |12719|9:9|deleted| (-6) vcld, main (line: 341)
89
90 2010-04-01 15:37:24|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 2 seconds before making next SSH attempt
91 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 2/3: executing SSH command on esx35-hypervisor-1:
92 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'netstat -an' 2>&1
93
94 |12719|9:9|deleted| ---- WARNING ----
95 |12719|9:9|deleted| 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 2/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
96 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not found
97 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
98 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
99 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
100 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
101 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
102 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
103 |12719|9:9|deleted| (-6) vcld, main (line: 341)
104
105 2010-04-01 15:37:26|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 15 seconds before making next SSH attempt
106 2010-04-01 15:37:29|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:37:29
107 2010-04-01 15:37:34|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:37:34
108 2010-04-01 15:37:39|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:37:39
109 2010-04-01 15:37:39|5754|10:10|new|utils.pm:reservation_being_processed(9634)|computerloadlog 'begin' entry does NOT exist for reservation 10
110 2010-04-01 15:37:39|5754|10:10|new|utils.pm:reservation_being_processed(9686)|reservation is NOT currently being processed
111 2010-04-01 15:37:39|5754|10:10|new|vcld:main(276)|reservation 10 is NOT already being processed
112
113 |5754|10:10|new| ---- WARNING ----
114 |5754|10:10|new| 2010-04-01 15:37:39|5754|10:10|new|utils.pm:get_request_info(5218)|preferredimageid is not set for computer id=3
115 |5754|10:10|new| ( 0) utils.pm, notify (line: 737)
116 |5754|10:10|new| (-1) utils.pm, get_request_info (line: 5218)
117 |5754|10:10|new| (-2) vcld, main (line: 280)
118
119 2010-04-01 15:37:39|5754|10:10|new|utils.pm:get_request_info(5354)|standalone affiliation found: Local
120 2010-04-01 15:37:39|5754|10:10|new|utils.pm:get_management_node_info(6815)|management node info retrieved from database for colbert
121 2010-04-01 15:37:39|5754|10:10|new|vcld:main(281)|retrieved request information from database
122 2010-04-01 15:37:39|5754|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 10
123 2010-04-01 15:37:39|5754|10:10|new|utils.pm:update_request_state(2186)|request 10 state updated to: pending, laststate to: new
124 2010-04-01 15:37:39|5754|10:10|new|utils.pm:insertloadlog(4710)|inserted computer=3, begin, beginning to process, state is new
125 2010-04-01 15:37:39|5754|10:10|new|vcld:make_new_child(509)|loaded VCL::new module
126 2010-04-01 15:37:39|12726|10:10|new|vcld:make_new_child(582)|vcld environment variable set to 0 for this process
127 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor called, class=VCL::new
128 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::new object created
129 2010-04-01 15:37:39|5754|10:10|new|vcld:make_new_child(566)|current number of forked kids: 2
130 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(85)|obtained a database handle for this state process, stored as $ENV{dbh}
131 2010-04-01 15:37:39|12726|10:10|new|State.pm:check_image_os(820)|no corrections need to be made, not an imaging request, returning 1
132 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 10
133 2010-04-01 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7863)|reservation count: 1
134 2010-04-01 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7882)|PARENTIMAGE: 1
135 2010-04-01 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7883)|SUBIMAGE: 0
136 2010-04-01 15:37:39|12726|10:10|new|utils.pm:rename_vcld_process(7901)|renamed process to 'vcld VCL::new 10:10 new'
137 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 10
138 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 10
139 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(121)|attempting to load provisioning module: VCL::Module::Provisioning::esx
140 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx vmware toolkit root path found: /usr/lib/vmware-vcli/apps
141 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(140)|vmware ESX module initialized
142 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(127)|VCL::Module::Provisioning::esx module loaded
143 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor called, class=VCL::Module::Provisioning::esx
144 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::Provisioning::esx object created
145 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(138)|esx vmware toolkit root path found: /usr/lib/vmware-vcli/apps
146 2010-04-01 15:37:39|12726|10:10|new|esx.pm:initialize(140)|vmware ESX module initialized
147 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(131)|VCL::Module::Provisioning::esx provisioner object created
148 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(145)|attempting to load OS module: VCL::Module::OS::Linux
149 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(151)|VCL::Module::OS::Linux module loaded
150 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(132)|constructor called, class=VCL::Module::OS::Linux
151 2010-04-01 15:37:39|12726|10:10|new|Module.pm:new(154)|VCL::Module::OS::Linux object created
152 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(154)|VCL::Module::OS::Linux OS object created
153 2010-04-01 15:37:39|12726|10:10|new|State.pm:initialize(169)|returning 1
154 2010-04-01 15:37:39|12726|10:10|new|vcld:make_new_child(591)|VCL::new object created and initialized
155 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 10
156 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting to retrieve current state of computer esx3-basewinxp-v0 from the database
157 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved current state of computer esx3-basewinxp-v0 from the database: available
158 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure updated: $self->request_data->{reservation}{10}{computer}{state}{name}
159 |12726|10:10|new| computer_state_name = available
160
161 |12726|10:10|new| ---- WARNING ----
162 |12726|10:10|new| 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_id: $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
163 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
164 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
165 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
166 |12726|10:10|new| (-3) new.pm, process (line: 106)
167 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
168 |12726|10:10|new| (-5) vcld, main (line: 341)
169
170
171 |12726|10:10|new| ---- WARNING ----
172 |12726|10:10|new| 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_name: $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
173 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
174 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
175 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
176 |12726|10:10|new| (-3) new.pm, process (line: 107)
177 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
178 |12726|10:10|new| (-5) vcld, main (line: 341)
179
180 2010-04-01 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment variable: 1
181 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(134)|reservation is parent = 1
182 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(135)|preload only = 0
183 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(136)|originating request state = new
184 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(137)|originating request laststate = new
185 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(138)|originating computer state = available
186 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(139)|originating computer type = virtualmachine
187 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 10
188 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting to retrieve current state of computer esx3-basewinxp-v0 from the database
189 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved current state of computer esx3-basewinxp-v0 from the database: available
190 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure updated: $self->request_data->{reservation}{10}{computer}{state}{name}
191 |12726|10:10|new| computer_state_name = available
192
193 |12726|10:10|new| ---- WARNING ----
194 |12726|10:10|new| 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_id: $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
195 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
196 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
197 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
198 |12726|10:10|new| (-3) new.pm, computer_not_being_used (line: 750)
199 |12726|10:10|new| (-4) new.pm, process (line: 170)
200 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
201 |12726|10:10|new| (-6) vcld, main (line: 341)
202
203
204 |12726|10:10|new| ---- WARNING ----
205 |12726|10:10|new| 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_name: $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
206 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
207 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
208 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
209 |12726|10:10|new| (-3) new.pm, computer_not_being_used (line: 751)
210 |12726|10:10|new| (-4) new.pm, process (line: 170)
211 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
212 |12726|10:10|new| (-6) vcld, main (line: 341)
213
214 2010-04-01 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment variable: 1
215 2010-04-01 15:37:39|12726|10:10|new|new.pm:computer_not_being_used(788)|esx3-basewinxp-v0 state is available
216 2010-04-01 15:37:39|12726|10:10|new|new.pm:computer_not_being_used(798)|esx3-basewinxp-v0 is available, its state is available
217 2010-04-01 15:37:39|12726|10:10|new|new.pm:process(170)|esx3-basewinxp-v0 is not being used
218 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:is_parent_reservation(854)|returning true: parent reservation ID for this request: 10
219 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting to retrieve current state of computer esx3-basewinxp-v0 from the database
220 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved current state of computer esx3-basewinxp-v0 from the database: available
221 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure updated: $self->request_data->{reservation}{10}{computer}{state}{name}
222 |12726|10:10|new| computer_state_name = available
223
224 |12726|10:10|new| ---- WARNING ----
225 |12726|10:10|new| 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_id: $self->request_data->{reservation}{10}{computer}{preferredimage}{id}
226 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
227 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
228 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
229 |12726|10:10|new| (-3) new.pm, reload_image (line: 480)
230 |12726|10:10|new| (-4) new.pm, process (line: 266)
231 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
232 |12726|10:10|new| (-6) vcld, main (line: 341)
233
234
235 |12726|10:10|new| ---- WARNING ----
236 |12726|10:10|new| 2010-04-01 15:37:39|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_preferredimage_name: $self->request_data->{reservation}{10}{computer}{preferredimage}{name}
237 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
238 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
239 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
240 |12726|10:10|new| (-3) new.pm, reload_image (line: 481)
241 |12726|10:10|new| (-4) new.pm, process (line: 266)
242 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
243 |12726|10:10|new| (-6) vcld, main (line: 341)
244
245 2010-04-01 15:37:39|12726|10:10|new|utils.pm:get_management_node_id(9364)|environment variable: 1
246 2010-04-01 15:37:39|12726|10:10|new|new.pm:reload_image(514)|calling VCL::Module::Provisioning::esx->node_status()
247 2010-04-01 15:37:39|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted computer=3, statuscheck, checking status of node
248 $VAR1 = bless( {
249 'data' => bless( do{\(my $o = 1)}, 'VCL::DataStructure' ),
250 'os' => bless( {
251 'provisioner' => $VAR1,
252 'data' => $VAR1->{'data'}
253 }, 'VCL::Module::OS::Linux' )
254 }, 'VCL::Module::Provisioning::esx' );
255 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(787)|Entering node_status, checking status of esx3-basewinxp-v0
256 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(788)|request_for_imaging: 0
257 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(789)|requeseted image name: esx3-basewinxp-v0
258 2010-04-01 15:37:39|12726|10:10|new|esx.pm:node_status(819)|checking if esx3-basewinxp-v0 is pingable
259 2010-04-01 15:37:40|12726|10:10|new|esx.pm:node_status(825)|esx3-basewinxp-v0 is not pingable (0)
260 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(560)|node_status returned a scalar: RELOAD
261 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(602)|node status is RELOAD, esx3-basewinxp-v0 will be reloaded
262 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted computer=3, loadimageblade, esx3-basewinxp-v0 must be reloaded with esx3-basewinxp-v0
263 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(616)|calling VCL::Module::Provisioning::esx->does_image_exist()
264 2010-04-01 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing SSH command on 192.168.129.37:
265 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x 192.168.129.37 'ls -1 /install/image/vmware_images/golden 2>&1' 2>&1
266 2010-04-01 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command output:
267 |12726|10:10|new| esx3-basewinxp-v0
268 2010-04-01 15:37:40|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH command executed on 192.168.129.37, returning (0, "esx3-basewinxp-v0")
269 2010-04-01 15:37:40|12726|10:10|new|esx.pm:does_image_exist(917)|image esx3-basewinxp-v0 exists
270 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(619)|esx3-basewinxp-v0 exists on this management node
271 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted computer=3, doesimageexists, confirmed image exists
272 2010-04-01 15:37:40|12726|10:10|new|utils.pm:update_computer_state(2228)|computer 3 state updated to: reloading
273 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(652)|computer esx3-basewinxp-v0 state set to reloading
274 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted computer=3, info, computer state updated to reloading
275 2010-04-01 15:37:40|12726|10:10|new|new.pm:reload_image(663)|calling VCL::Module::Provisioning::esx->load() subroutine
276 2010-04-01 15:37:40|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted computer=3, info, calling VCL::Module::Provisioning::esx->load() subroutine
277 2010-04-01 15:37:40|12726|10:10|new|esx.pm:load(165)|****************************************************
278
279 |12726|10:10|new| ---- WARNING ----
280 |12726|10:10|new| 2010-04-01 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_eth0_mac_address: $self->request_data->{reservation}{10}{computer}{eth0macaddress}
281 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
282 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
283 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
284 |12726|10:10|new| (-3) esx.pm, load (line: 181)
285 |12726|10:10|new| (-4) new.pm, reload_image (line: 665)
286 |12726|10:10|new| (-5) new.pm, process (line: 266)
287 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
288
289
290 |12726|10:10|new| ---- WARNING ----
291 |12726|10:10|new| 2010-04-01 15:37:40|12726|10:10|new|DataStructure.pm:_automethod(675)|corresponding data has not been initialized for get_computer_eth1_mac_address: $self->request_data->{reservation}{10}{computer}{eth1macaddress}
292 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
293 |12726|10:10|new| (-1) DataStructure.pm, _automethod (line: 675)
294 |12726|10:10|new| (-2) Autoload.pm, __ANON__ (line: 80)
295 |12726|10:10|new| (-3) esx.pm, load (line: 182)
296 |12726|10:10|new| (-4) new.pm, reload_image (line: 665)
297 |12726|10:10|new| (-5) new.pm, process (line: 266)
298 |12726|10:10|new| (-6) vcld, make_new_child (line: 594)
299
300 2010-04-01 15:37:40|12726|10:10|new|esx.pm:load(193)|Calling get_vmware_host_info
301 No recipient addresses found in header
302 2010-04-01 15:37:41|12726|10:10|new|utils.pm:mail(1348)|SUCCESS -- Sending mail To: , PROBLEM -- esx.pm
303
304 |12726|10:10|new| ---- CRITICAL ----
305 |12726|10:10|new| 2010-04-01 15:37:40|12726|10:10|new|esx.pm:get_vmware_host_info(1012)|AMIT DEBUGGING: esx35-hypervisor-1.odu.edu
306 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
307 |12726|10:10|new| (-1) esx.pm, get_vmware_host_info (line: 1012)
308 |12726|10:10|new| (-2) esx.pm, load (line: 194)
309 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
310 |12726|10:10|new| (-4) new.pm, process (line: 266)
311 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
312 |12726|10:10|new| (-6) vcld, main (line: 341)
313
314 2010-04-01 15:37:41|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 3/3: executing SSH command on esx35-hypervisor-1:
315 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'netstat -an' 2>&1
316
317 |12719|9:9|deleted| ---- WARNING ----
318 |12719|9:9|deleted| 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 3/3: failed to execute SSH command on esx35-hypervisor-1: netstat -an, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
319 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not found
320 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
321 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
322 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
323 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
324 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
325 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
326 |12719|9:9|deleted| (-6) vcld, main (line: 341)
327
328
329 |12719|9:9|deleted| ---- WARNING ----
330 |12719|9:9|deleted| 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed to run SSH command after 3 attempts, command: /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'netstat -an' 2>&1, exit status: 255, output:
331 |12719|9:9|deleted| ssh output (netstat -a...): ash: netstat: not found
332 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
333 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
334 |12719|9:9|deleted| (-2) Linux.pm, is_connected (line: 965)
335 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 837)
336 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
337 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
338 |12719|9:9|deleted| (-6) vcld, main (line: 341)
339
340 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing SSH command on esx35-hypervisor-1:
341 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f /etc/ssh/external_sshd_config"' 2>&1
342
343 |12719|9:9|deleted| ---- WARNING ----
344 |12719|9:9|deleted| 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 1/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
345 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not found
346 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
347 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
348 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line: 913)
349 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
350 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
351 |12719|9:9|deleted| (-5) reclaim.pm, process (line: 173)
352 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
353
354 2010-04-01 15:37:42|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 2 seconds before making next SSH attempt
355 2010-04-01 15:37:43|12726|10:10|new|esx.pm:get_vmware_host_info(1019)|host info output for esx35-hypervisor-1
356 |12726|10:10|new| Host Information
357 |12726|10:10|new| Host Name: esx35-hypervisor-1.odu.edu
358 2010-04-01 15:37:43|12726|10:10|new|esx.pm:get_vmware_host_info(1025)|found hostname_value= esx35-hypervisor-1.odu.edu
359 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(197)|Collected esx35-hypervisor-1.odu.edu for vmware host name
360 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(210)|DATASTORE IP is 192.168.129.37 and DATASTORE_SHARE_PATH is /install/image/vmware_images
361 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(211)|Entered ESX module, loading esx3-basewinxp-v0 on esx3-basewinxp-v0 (on esx35-hypervisor-1.odu.edu) for reservation 10
362 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(212)|Datastore: 192.168.129.37:/install/image/vmware_images
363 2010-04-01 15:37:43|12726|10:10|new|esx.pm:load(223)|VM info command: /usr/lib/vmware-vcli/apps/vm/vminfo.pl --server 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --username root --password 'xxxxxxxxxxx'
364 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 2/3: executing SSH command on esx35-hypervisor-1:
365 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f /etc/ssh/external_sshd_config"' 2>&1
366 2010-04-01 15:37:44|12726|10:10|new|esx.pm:load(226)|VM info output: Virtual Machine esx3-basewinxp-v0 not found.
367 2010-04-01 15:37:44|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing SSH command on 192.168.129.37:
368 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x 192.168.129.37 'rm -rf /install/image/vmware_images/inuse/esx3-basewinxp-v0' 2>&1
369 2010-04-01 15:37:44|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:37:44
370
371 |12719|9:9|deleted| ---- WARNING ----
372 |12719|9:9|deleted| 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 2/3: failed to execute SSH command on esx35-hypervisor-1: pkill -fx "/usr/sbin/sshd -f /etc/ssh/external_sshd_config", exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
373 |12719|9:9|deleted| ssh output (pkill -fx ...): ash: pkill: not found
374 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
375 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
376 |12719|9:9|deleted| (-2) Linux.pm, stop_external_sshd (line: 913)
377 |12719|9:9|deleted| (-3) Linux.pm, revoke_access (line: 884)
378 |12719|9:9|deleted| (-4) Linux.pm, sanitize (line: 844)
379 |12719|9:9|deleted| (-5) reclaim.pm, process (line: 173)
380 |12719|9:9|deleted| (-6) vcld, make_new_child (line: 594)
381
382 2010-04-01 15:37:44|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 15 seconds before making next SSH attempt
383 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command output:
384 |12726|10:10|new| none
385 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH command executed on 192.168.129.37, returning (0, "none")
386 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(262)|Removed old vm folder
387 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing SSH command on 192.168.129.37:
388 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x 192.168.129.37 'mkdir /install/image/vmware_images/inuse/esx3-basewinxp-v0' 2>&1
389 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command output:
390 |12726|10:10|new| none
391 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH command executed on 192.168.129.37, returning (0, "none")
392 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing SSH command on 192.168.129.37:
393 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x 192.168.129.37 'cp /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk' 2>&1
394 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command output:
395 |12726|10:10|new| none
396 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH command executed on 192.168.129.37, returning (0, "none")
397 2010-04-01 15:37:45|12726|10:10|new|esx.pm:load(278)|COPIED VMDK SUCCESSFULLY
398 2010-04-01 15:37:45|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing SSH command on 192.168.129.37:
399 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x 192.168.129.37 'cp /install/image/vmware_images/golden/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0-flat.vmdk' 2>&1
400 2010-04-01 15:37:49|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:37:49
401 2010-04-01 15:37:54|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:37:54
402 2010-04-01 15:37:59|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 3/3: executing SSH command on esx35-hypervisor-1:
403 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f /etc/ssh/external_sshd_config"' 2>&1
404 2010-04-01 15:37:59|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:37:59
405 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6262)|run_ssh_command output:
406 |12719|9:9|deleted| ash: pkill: not found
407 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6272)|SSH command executed on esx35-hypervisor-1, command:
408 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'pkill -fx "/usr/sbin/sshd -f /etc/ssh/external_sshd_config"' 2>&1
409 |12719|9:9|deleted| returning (127, "ash: pkill: not found")
410 2010-04-01 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(916)|output detected: ash: pkill: not found
411 2010-04-01 15:38:00|12719|9:9|deleted|Linux.pm:stop_external_sshd(921)|ext_sshd on esx35-hypervisor-1 stopped
412 2010-04-01 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(884)|stopped external sshd
413 2010-04-01 15:38:00|12719|9:9|deleted|Linux.pm:revoke_access(888)|access has been revoked to esx35-hypervisor-1
414 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing SSH command on esx35-hypervisor-1:
415 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
416
417 |12719|9:9|deleted| ---- WARNING ----
418 |12719|9:9|deleted| 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 1/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel admin, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
419 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/passwd'
420 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/shadow'
421 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/group'
422 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/gshadow'
423 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
424 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
425 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
426 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
427 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
428 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
429 |12719|9:9|deleted| (-6) vcld, main (line: 341)
430
431 2010-04-01 15:38:00|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 2 seconds before making next SSH attempt
432 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 2/3: executing SSH command on esx35-hypervisor-1:
433 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
434
435 |12719|9:9|deleted| ---- WARNING ----
436 |12719|9:9|deleted| 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 2/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel admin, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
437 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/passwd'
438 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/shadow'
439 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/group'
440 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/gshadow'
441 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
442 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
443 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
444 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
445 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
446 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
447 |12719|9:9|deleted| (-6) vcld, main (line: 341)
448
449 2010-04-01 15:38:02|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 15 seconds before making next SSH attempt
450 2010-04-01 15:38:04|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:38:04
451 2010-04-01 15:38:09|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:38:09
452 2010-04-01 15:38:14|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:38:14
453 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 3/3: executing SSH command on esx35-hypervisor-1:
454 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
455
456 |12719|9:9|deleted| ---- WARNING ----
457 |12719|9:9|deleted| 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 3/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel admin, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
458 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/passwd'
459 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/shadow'
460 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/group'
461 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/gshadow'
462 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
463 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
464 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
465 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
466 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
467 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
468 |12719|9:9|deleted| (-6) vcld, main (line: 341)
469
470
471 |12719|9:9|deleted| ---- WARNING ----
472 |12719|9:9|deleted| 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed to run SSH command after 3 attempts, command: /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1, exit status: 255, output:
473 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/passwd'
474 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/shadow'
475 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/group'
476 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/gshadow'
477 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
478 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
479 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 543)
480 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
481 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
482 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
483 |12719|9:9|deleted| (-6) vcld, main (line: 341)
484
485 2010-04-01 15:38:17|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing SSH command on esx35-hypervisor-1:
486 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
487
488 |12719|9:9|deleted| ---- WARNING ----
489 |12719|9:9|deleted| 2010-04-01 15:38:18|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 1/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel admin, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
490 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/passwd'
491 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/shadow'
492 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/group'
493 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/gshadow'
494 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
495 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
496 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
497 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
498 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
499 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
500 |12719|9:9|deleted| (-6) vcld, main (line: 341)
501
502 2010-04-01 15:38:18|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 2 seconds before making next SSH attempt
503 2010-04-01 15:38:19|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:38:19
504 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 2/3: executing SSH command on esx35-hypervisor-1:
505 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
506
507 |12719|9:9|deleted| ---- WARNING ----
508 |12719|9:9|deleted| 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 2/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel admin, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
509 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/passwd'
510 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/shadow'
511 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/group'
512 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/gshadow'
513 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
514 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
515 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
516 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
517 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
518 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
519 |12719|9:9|deleted| (-6) vcld, main (line: 341)
520
521 2010-04-01 15:38:20|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 15 seconds before making next SSH attempt
522 2010-04-01 15:38:25|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:38:24
523 2010-04-01 15:38:30|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:38:30
524 2010-04-01 15:38:35|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:38:35
525 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 3/3: executing SSH command on esx35-hypervisor-1:
526 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1
527
528 |12719|9:9|deleted| ---- WARNING ----
529 |12719|9:9|deleted| 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 3/3: failed to execute SSH command on esx35-hypervisor-1: /usr/sbin/userdel admin, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
530 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/passwd'
531 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/shadow'
532 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/group'
533 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/gshadow'
534 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
535 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
536 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
537 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
538 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
539 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
540 |12719|9:9|deleted| (-6) vcld, main (line: 341)
541
542
543 |12719|9:9|deleted| ---- WARNING ----
544 |12719|9:9|deleted| 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed to run SSH command after 3 attempts, command: /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 '/usr/sbin/userdel admin' 2>&1, exit status: 255, output:
545 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/passwd'
546 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/shadow'
547 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/group'
548 |12719|9:9|deleted| ssh output (/usr/sbin/...): userdel: Can't find 'admin' in '/etc/gshadow'
549 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
550 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
551 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 553)
552 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
553 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
554 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
555 |12719|9:9|deleted| (-6) vcld, main (line: 341)
556
557 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing SSH command on esx35-hypervisor-1:
558 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config' 2>&1
559
560 |12719|9:9|deleted| ---- WARNING ----
561 |12719|9:9|deleted| 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
562 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/ssh/external_sshd_config: No such file or directory
563 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
564 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
565 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
566 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
567 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
568 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
569 |12719|9:9|deleted| (-6) vcld, main (line: 341)
570
571 2010-04-01 15:38:35|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 2 seconds before making next SSH attempt
572 2010-04-01 15:38:37|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 2/3: executing SSH command on esx35-hypervisor-1:
573 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config' 2>&1
574
575 |12719|9:9|deleted| ---- WARNING ----
576 |12719|9:9|deleted| 2010-04-01 15:38:38|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
577 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/ssh/external_sshd_config: No such file or directory
578 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
579 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
580 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
581 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
582 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
583 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
584 |12719|9:9|deleted| (-6) vcld, main (line: 341)
585
586 2010-04-01 15:38:38|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 15 seconds before making next SSH attempt
587 2010-04-01 15:38:40|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:38:40
588 2010-04-01 15:38:45|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:38:45
589 2010-04-01 15:38:50|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:38:50
590 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 3/3: executing SSH command on esx35-hypervisor-1:
591 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config' 2>&1
592
593 |12719|9:9|deleted| ---- WARNING ----
594 |12719|9:9|deleted| 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
595 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/ssh/external_sshd_config: No such file or directory
596 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
597 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
598 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
599 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
600 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
601 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
602 |12719|9:9|deleted| (-6) vcld, main (line: 341)
603
604
605 |12719|9:9|deleted| ---- WARNING ----
606 |12719|9:9|deleted| 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed to run SSH command after 3 attempts, command: /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^AllowUsers .*/d" /etc/ssh/external_sshd_config' 2>&1, exit status: 255, output:
607 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/ssh/external_sshd_config: No such file or directory
608 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
609 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
610 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 562)
611 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
612 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
613 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
614 |12719|9:9|deleted| (-6) vcld, main (line: 341)
615
616 No recipient addresses found in header
617 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:mail(1348)|SUCCESS -- Sending mail To: , PROBLEM -- Linux.pm
618
619 |12719|9:9|deleted| ---- CRITICAL ----
620 |12719|9:9|deleted| 2010-04-01 15:38:53|12719|9:9|deleted|Linux.pm:delete_user(566)|failed to add AllowUsers admin to external_sshd_config
621 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
622 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 566)
623 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
624 |12719|9:9|deleted| (-3) reclaim.pm, process (line: 173)
625 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
626 |12719|9:9|deleted| (-5) vcld, main (line: 341)
627
628 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6180)|executing SSH command on esx35-hypervisor-1:
629 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
630
631 |12719|9:9|deleted| ---- WARNING ----
632 |12719|9:9|deleted| 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 1/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
633 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers: No such file or directory
634 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
635 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
636 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
637 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
638 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
639 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
640 |12719|9:9|deleted| (-6) vcld, main (line: 341)
641
642 2010-04-01 15:38:53|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 2 seconds before making next SSH attempt
643 2010-04-01 15:38:55|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:38:55
644 2010-04-01 15:38:55|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 2/3: executing SSH command on esx35-hypervisor-1:
645 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
646
647 |12719|9:9|deleted| ---- WARNING ----
648 |12719|9:9|deleted| 2010-04-01 15:38:56|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 2/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
649 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers: No such file or directory
650 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
651 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
652 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
653 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
654 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
655 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
656 |12719|9:9|deleted| (-6) vcld, main (line: 341)
657
658 2010-04-01 15:38:56|12719|9:9|deleted|utils.pm:run_ssh_command(6172)|sleeping for 15 seconds before making next SSH attempt
659 2010-04-01 15:39:00|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:00
660 2010-04-01 15:39:05|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:05
661 2010-04-01 15:39:10|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:10
662 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6184)|attempt 3/3: executing SSH command on esx35-hypervisor-1:
663 |12719|9:9|deleted| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1
664
665 |12719|9:9|deleted| ---- WARNING ----
666 |12719|9:9|deleted| 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6252)|attempt 3/3: failed to execute SSH command on esx35-hypervisor-1: sed -ie "/^admin .*/d" /etc/sudoers, exit status: 255, SSH exits with the exit status of the remote command or with 255 if an error occurred, output:
667 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers: No such file or directory
668 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
669 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6252)
670 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
671 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
672 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
673 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
674 |12719|9:9|deleted| (-6) vcld, main (line: 341)
675
676
677 |12719|9:9|deleted| ---- WARNING ----
678 |12719|9:9|deleted| 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:run_ssh_command(6285)|failed to run SSH command after 3 attempts, command: /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x esx35-hypervisor-1 'sed -ie "/^admin .*/d" /etc/sudoers' 2>&1, exit status: 255, output:
679 |12719|9:9|deleted| ssh output (sed -ie "/...): sed: /etc/sudoers: No such file or directory
680 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
681 |12719|9:9|deleted| (-1) utils.pm, run_ssh_command (line: 6285)
682 |12719|9:9|deleted| (-2) Linux.pm, delete_user (line: 574)
683 |12719|9:9|deleted| (-3) Linux.pm, sanitize (line: 851)
684 |12719|9:9|deleted| (-4) reclaim.pm, process (line: 173)
685 |12719|9:9|deleted| (-5) vcld, make_new_child (line: 594)
686 |12719|9:9|deleted| (-6) vcld, main (line: 341)
687
688 No recipient addresses found in header
689 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:mail(1348)|SUCCESS -- Sending mail To: , PROBLEM -- Linux.pm
690
691 |12719|9:9|deleted| ---- CRITICAL ----
692 |12719|9:9|deleted| 2010-04-01 15:39:11|12719|9:9|deleted|Linux.pm:delete_user(578)|failed to clear admin from /etc/sudoers
693 |12719|9:9|deleted| ( 0) utils.pm, notify (line: 737)
694 |12719|9:9|deleted| (-1) Linux.pm, delete_user (line: 578)
695 |12719|9:9|deleted| (-2) Linux.pm, sanitize (line: 851)
696 |12719|9:9|deleted| (-3) reclaim.pm, process (line: 173)
697 |12719|9:9|deleted| (-4) vcld, make_new_child (line: 594)
698 |12719|9:9|deleted| (-5) vcld, main (line: 341)
699
700 2010-04-01 15:39:11|12719|9:9|deleted|Linux.pm:sanitize(852)|users have been deleted from esx35-hypervisor-1
701 2010-04-01 15:39:11|12719|9:9|deleted|reclaim.pm:process(173)|esx35-hypervisor-1 has been sanitized
702 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8018)|called from VCL::reclaim::process(271)
703 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8054)|parent: parent reservation ID for this request: 9
704 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog entry not inserted, reservation has been deleted
705 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:update_request_state(2186)|request 9 state updated to: complete, laststate to: deleted
706 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8078)|req=9: request state changed: deleted->complete, laststate: reserved->deleted
707 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog entry not inserted, reservation has been deleted
708 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog entry not inserted, reservation has been deleted
709 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:update_computer_state(2228)|computer 2 state updated to: available
710 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8096)|req=9: computer esx35-hypervisor-1 state changed: available->available
711 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8106)|req=9: log table id=10 will not be updated
712 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:insertloadlog(4683)|computerloadlog entry not inserted, reservation has been deleted
713 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:switch_state(8122)|req=9: process exiting
714 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(905)|destructor called, ref($self)=VCL::reclaim
715 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:delete_computerloadlog_reservation(7551)|removing computerloadlog entries matching loadstate = begin
716 2010-04-01 15:39:11|12719|9:9|deleted|utils.pm:delete_computerloadlog_reservation(7598)|deleted rows from computerloadlog for reservation id=9
717 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(912)|removed computerloadlog rows with loadstate=begin for reservation
718 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(924)|number of database handles state process created: 1
719 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(933)|process has a database handle stored in $ENV{dbh}, attempting disconnect
720 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(935)|$ENV{dbh}: database disconnect successful
721 2010-04-01 15:39:11|12719|9:9|deleted|State.pm:DESTROY(949)|VCL::reclaim process 12719 exiting
722 2010-04-01 15:39:11|5754|vcld:REAPER(744)|VCL process exited for reservation 9
723 2010-04-01 15:39:11|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:11
724 2010-04-01 15:39:11|5754|9:9|complete|vcld:main(250)|request deleted
725 2010-04-01 15:39:16|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:16
726 2010-04-01 15:39:21|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:21
727 2010-04-01 15:39:26|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:26
728 2010-04-01 15:39:31|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:31
729 2010-04-01 15:39:36|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:36
730 2010-04-01 15:39:42|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:42
731 2010-04-01 15:39:47|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:47
732 2010-04-01 15:39:52|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:52
733 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command output:
734 |12726|10:10|new| none
735 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH command executed on 192.168.129.37, returning (0, "none")
736 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6180)|executing SSH command on 192.168.129.37:
737 |12726|10:10|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x 192.168.129.37 'grep adapterType /install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmdk 2>&1' 2>&1
738 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6262)|run_ssh_command output:
739 |12726|10:10|new| ddb.adapterType = "buslogic"
740 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_ssh_command(6276)|SSH command executed on 192.168.129.37, returning (0, "ddb.adapterType = "buslogic"")
741 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(311)|adapter= buslogic
742 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(342)|eth0 MAC address set for vcl assigned
743 Use of uninitialized value in concatenation (.) or string at
744 /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 343 (#1)
745 (W uninitialized) An undefined value was used as if it were already
746 defined. It was interpreted as a "" or a 0, but maybe it was a mistake.
747 To suppress this warning assign a defined value to your variables.
748
749 To help you figure out what was undefined, perl tells you what operation
750 you used the undefined value in. Note, however, that perl optimizes your
751 program and the operation displayed in the warning may not necessarily
752 appear literally in your program. For example, "that $foo" is
753 usually optimized into "that " . $foo, and the warning will refer to
754 the concatenation (.) operator, even though there is no . in your
755 program.
756
757
758 |12726|10:10|new| ---- WARNING ----
759 |12726|10:10|new| 2010-04-01 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized value in concatenation (.) or string at /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 343.
760 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
761 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
762 |12726|10:10|new| (-2) esx.pm, load (line: 343)
763 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
764 |12726|10:10|new| (-4) new.pm, process (line: 266)
765 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
766 |12726|10:10|new| (-6) vcld, main (line: 341)
767
768
769 |12726|10:10|new| ---- WARNING ----
770 |12726|10:10|new| 2010-04-01 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized value in concatenation (.) or string at /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 353.
771 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
772 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
773 |12726|10:10|new| (-2) esx.pm, load (line: 353)
774 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
775 |12726|10:10|new| (-4) new.pm, process (line: 266)
776 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
777 |12726|10:10|new| (-6) vcld, main (line: 341)
778
779 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(353)|eth1 MAC address set for vcl assigned
780
781 |12726|10:10|new| ---- WARNING ----
782 |12726|10:10|new| 2010-04-01 15:39:55|12726|10:10|new|vcld:warning_handler(636)|Use of uninitialized value in concatenation (.) or string at /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 354.
783 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
784 |12726|10:10|new| (-1) vcld, warning_handler (line: 636)
785 |12726|10:10|new| (-2) esx.pm, load (line: 354)
786 |12726|10:10|new| (-3) new.pm, reload_image (line: 665)
787 |12726|10:10|new| (-4) new.pm, process (line: 266)
788 |12726|10:10|new| (-5) vcld, make_new_child (line: 594)
789 |12726|10:10|new| (-6) vcld, main (line: 341)
790
791 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(376)|wrote vmxarray to /tmp/esx3-basewinxp-v0.vmx
792 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_scp_command(6403)|attempt 1/3: executing SCP command: /usr/bin/scp -B -i /etc/vcl/vcl.key -P 22 -p -r /tmp/esx3-basewinxp-v0.vmx 192.168.129.37:/install/image/vmware_images/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmx 2>&1
793 2010-04-01 15:39:55|12726|10:10|new|utils.pm:run_scp_command(6453)|scp successful: attempt 1/3, exit status: 0, output: none
794 2010-04-01 15:39:55|12726|10:10|new|esx.pm:load(401)|Register Command: /usr/lib/vmware-vcli/apps/vm/vmregister.pl --server 'esx35-hypervisor-1' --username root --password 'xxxxxxxxxxx' --vmxpath '[VCL]/inuse/esx3-basewinxp-v0/esx3-basewinxp-v0.vmx' --operation register --vmname esx3-basewinxp-v0 --pool Resources --hostname 'esx35-hypervisor-1.odu.edu' --datacenter 'ha-datacenter'
795 2010-04-01 15:39:57|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:39:57
796 2010-04-01 15:39:57|12726|10:10|new|esx.pm:load(404)|Registered:
797 |12726|10:10|new| SOAP Fault:
798 |12726|10:10|new| -----------
799 |12726|10:10|new| Fault string: fault.RestrictedVersion.summary
800 |12726|10:10|new| Fault detail: RestrictedVersionFault
801 2010-04-01 15:39:57|12726|10:10|new|esx.pm:load(413)|Power on command: /usr/lib/vmware-vcli/apps/vm/vmcontrol.pl --server 'esx35-hypervisor-1' --vmname esx3-basewinxp-v0 --operation poweron --username root --password 'xxxxxxxxxxx'
802 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(416)|Powered on: Virtual Machine esx3-basewinxp-v0 not found.
803 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(505)|IP is known for esx3-basewinxp-v0
804 2010-04-01 15:39:59|12726|10:10|new|esx.pm:load(511)|Waiting for ssh to come up on esx3-basewinxp-v0
805 2010-04-01 15:40:00|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 1
806 2010-04-01 15:40:02|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:02
807 2010-04-01 15:40:06|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 2
808 2010-04-01 15:40:07|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:07
809 2010-04-01 15:40:12|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:12
810 2010-04-01 15:40:12|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 3
811 2010-04-01 15:40:17|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:17
812 2010-04-01 15:40:18|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 4
813 2010-04-01 15:40:22|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:22
814 2010-04-01 15:40:24|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 5
815 2010-04-01 15:40:27|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:27
816 2010-04-01 15:40:30|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 6
817 2010-04-01 15:40:32|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:32
818 2010-04-01 15:40:36|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 7
819 2010-04-01 15:40:37|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:37
820 2010-04-01 15:40:42|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:42
821 2010-04-01 15:40:42|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 8
822 2010-04-01 15:40:47|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:47
823 2010-04-01 15:40:48|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 9
824 2010-04-01 15:40:52|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:52
825 2010-04-01 15:40:54|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 10
826 2010-04-01 15:40:57|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:40:57
827 2010-04-01 15:41:00|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 11
828 2010-04-01 15:41:02|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:02
829 2010-04-01 15:41:06|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 12
830 2010-04-01 15:41:07|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:07
831 2010-04-01 15:41:12|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:12
832 2010-04-01 15:41:12|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 13
833 2010-04-01 15:41:17|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:17
834 2010-04-01 15:41:18|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 14
835 2010-04-01 15:41:22|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:22
836 2010-04-01 15:41:24|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 15
837 2010-04-01 15:41:27|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:27
838 2010-04-01 15:41:30|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 16
839 2010-04-01 15:41:32|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:32
840 2010-04-01 15:41:36|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 17
841 2010-04-01 15:41:37|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:37
842 2010-04-01 15:41:42|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:42
843 2010-04-01 15:41:42|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 18
844 2010-04-01 15:41:47|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:47
845 2010-04-01 15:41:48|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 19
846 2010-04-01 15:41:52|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:52
847 2010-04-01 15:41:54|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 20
848 2010-04-01 15:41:57|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:41:57
849 2010-04-01 15:42:01|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 21
850 2010-04-01 15:42:02|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:02
851 2010-04-01 15:42:07|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 22
852 2010-04-01 15:42:07|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:07
853 2010-04-01 15:42:12|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:12
854 2010-04-01 15:42:13|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 23
855 2010-04-01 15:42:18|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:18
856 2010-04-01 15:42:19|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 24
857 2010-04-01 15:42:23|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:23
858 2010-04-01 15:42:25|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 25
859 2010-04-01 15:42:28|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:28
860 2010-04-01 15:42:31|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 26
861 2010-04-01 15:42:33|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:33
862 2010-04-01 15:42:37|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 27
863 2010-04-01 15:42:38|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:38
864 2010-04-01 15:42:43|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 28
865 2010-04-01 15:42:43|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:43
866 2010-04-01 15:42:48|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:48
867 2010-04-01 15:42:49|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 29
868 2010-04-01 15:42:53|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:53
869 2010-04-01 15:42:55|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 30
870 2010-04-01 15:42:58|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:42:58
871 2010-04-01 15:43:01|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 31
872 2010-04-01 15:43:03|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:03
873 2010-04-01 15:43:07|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 32
874 2010-04-01 15:43:08|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:08
875 2010-04-01 15:43:13|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 33
876 2010-04-01 15:43:13|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:13
877 2010-04-01 15:43:18|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:18
878 2010-04-01 15:43:19|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 34
879 2010-04-01 15:43:23|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:23
880 2010-04-01 15:43:25|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 35
881 2010-04-01 15:43:28|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:28
882 2010-04-01 15:43:31|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 36
883 2010-04-01 15:43:33|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:33
884 2010-04-01 15:43:37|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 37
885 2010-04-01 15:43:38|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:38
886 2010-04-01 15:43:43|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 38
887 2010-04-01 15:43:43|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:43
888 2010-04-01 15:43:48|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:48
889 2010-04-01 15:43:49|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 39
890 2010-04-01 15:43:53|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:53
891 2010-04-01 15:43:55|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 40
892 2010-04-01 15:43:58|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:43:58
893 2010-04-01 15:44:01|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 41
894 2010-04-01 15:44:03|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:03
895 2010-04-01 15:44:07|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 42
896 2010-04-01 15:44:08|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:08
897 2010-04-01 15:44:13|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 43
898 2010-04-01 15:44:13|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:13
899 2010-04-01 15:44:18|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:18
900 2010-04-01 15:44:19|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 44
901 2010-04-01 15:44:23|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:23
902 2010-04-01 15:44:25|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 45
903 2010-04-01 15:44:29|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:28
904 2010-04-01 15:44:31|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 46
905 2010-04-01 15:44:34|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:34
906 2010-04-01 15:44:37|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 47
907 2010-04-01 15:44:39|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:39
908 2010-04-01 15:44:43|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 48
909 2010-04-01 15:44:44|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:44
910 2010-04-01 15:44:49|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:49
911 2010-04-01 15:44:49|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 49
912 2010-04-01 15:44:54|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:54
913 2010-04-01 15:44:55|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 50
914 2010-04-01 15:44:59|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:44:59
915 2010-04-01 15:45:01|12726|10:10|new|esx.pm:load(528)|going to sleep 5 seconds, waiting for computer to start SSH. Try 51
916 2010-04-01 15:45:04|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:45:04
917 No recipient addresses found in header
918 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS -- Sending mail To: , PROBLEM -- esx.pm
919
920 |12726|10:10|new| ---- CRITICAL ----
921 |12726|10:10|new| 2010-04-01 15:45:07|12726|10:10|new|esx.pm:load(521)|waited acceptable amount of time for sshd to become active, please check esx3-basewinxp-v0 on esx35-hypervisor-1
922 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
923 |12726|10:10|new| (-1) esx.pm, load (line: 521)
924 |12726|10:10|new| (-2) new.pm, reload_image (line: 665)
925 |12726|10:10|new| (-3) new.pm, process (line: 266)
926 |12726|10:10|new| (-4) vcld, make_new_child (line: 594)
927 |12726|10:10|new| (-5) vcld, main (line: 341)
928
929
930 |12726|10:10|new| ---- WARNING ----
931 |12726|10:10|new| 2010-04-01 15:45:08|12726|10:10|new|new.pm:reload_image(670)|esx3-basewinxp-v0 failed to load on esx3-basewinxp-v0, returning
932 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
933 |12726|10:10|new| (-1) new.pm, reload_image (line: 670)
934 |12726|10:10|new| (-2) new.pm, process (line: 266)
935 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
936 |12726|10:10|new| (-4) vcld, main (line: 341)
937
938 2010-04-01 15:45:08|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted computer=3, loadimagefailed, esx3-basewinxp-v0 failed to load on esx3-basewinxp-v0
939
940 |12726|10:10|new| ---- WARNING ----
941 |12726|10:10|new| 2010-04-01 15:45:08|12726|10:10|new|new.pm:process(313)|failed to load esx3-basewinxp-v0 with esx3-basewinxp-v0
942 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
943 |12726|10:10|new| (-1) new.pm, process (line: 313)
944 |12726|10:10|new| (-2) vcld, make_new_child (line: 594)
945 |12726|10:10|new| (-3) vcld, main (line: 341)
946
947 2010-04-01 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1946)|attempting to retrieve current state of computer esx3-basewinxp-v0 from the database
948 2010-04-01 15:45:08|12726|10:10|new|DataStructure.pm:get_computer_state_name(1977)|retrieved current state of computer esx3-basewinxp-v0 from the database: reloading
949 2010-04-01 15:45:08|12726|10:10|new|DataStructure.pm:_automethod(697)|data structure updated: $self->request_data->{reservation}{10}{computer}{state}{name}
950 |12726|10:10|new| computer_state_name = reloading
951 No recipient addresses found in header
952 2010-04-01 15:45:08|12726|10:10|new|utils.pm:mail(1348)|SUCCESS -- Sending mail To: , PROBLEM -- State.pm
953
954 |12726|10:10|new| ---- CRITICAL ----
955 |12726|10:10|new| 2010-04-01 15:45:08|12726|10:10|new|State.pm:reservation_failed(290)|reservation failed on esx3-basewinxp-v0: process failed after trying to load or make available
956 |12726|10:10|new| ( 0) utils.pm, notify (line: 737)
957 |12726|10:10|new| (-1) State.pm, reservation_failed (line: 290)
958 |12726|10:10|new| (-2) new.pm, process (line: 316)
959 |12726|10:10|new| (-3) vcld, make_new_child (line: 594)
960 |12726|10:10|new| (-4) vcld, main (line: 341)
961
962 2010-04-01 15:45:08|12726|10:10|new|utils.pm:insertloadlog(4710)|inserted computer=3, failed, process failed after trying to load or make available
963 2010-04-01 15:45:08|12726|10:10|new|State.pm:reservation_failed(293)|inserted computerloadlog entry
964 2010-04-01 15:45:08|12726|10:10|new|State.pm:reservation_failed(301)|updated log ending value to 'failed', logid=11
965 2010-04-01 15:45:08|12726|10:10|new|utils.pm:update_computer_state(2228)|computer 3 state updated to: failed
966 2010-04-01 15:45:08|12726|10:10|new|State.pm:reservation_failed(312)|computer esx3-basewinxp-v0 (3) state set to failed
967 2010-04-01 15:45:08|12726|10:10|new|utils.pm:update_request_state(2186)|request 10 state updated to: failed, laststate to: new
968 2010-04-01 15:45:08|12726|10:10|new|State.pm:reservation_failed(325)|set request state to 'failed'/'new'
969 2010-04-01 15:45:08|12726|10:10|new|utils.pm:is_inblockrequest(6972)|zero rows were returned from database select
970 2010-04-01 15:45:08|12726|10:10|new|State.pm:reservation_failed(343)|esx3-basewinxp-v0 is NOT in blockcomputers table
971 2010-04-01 15:45:08|12726|10:10|new|State.pm:reservation_failed(346)|exiting 1
972 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(905)|destructor called, ref($self)=VCL::new
973 2010-04-01 15:45:08|12726|10:10|new|utils.pm:delete_computerloadlog_reservation(7551)|removing computerloadlog entries matching loadstate = begin
974 2010-04-01 15:45:08|12726|10:10|new|utils.pm:delete_computerloadlog_reservation(7598)|deleted rows from computerloadlog for reservation id=10
975 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(912)|removed computerloadlog rows with loadstate=begin for reservation
976 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(924)|number of database handles state process created: 1
977 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(933)|process has a database handle stored in $ENV{dbh}, attempting disconnect
978 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(935)|$ENV{dbh}: database disconnect successful
979 2010-04-01 15:45:08|12726|10:10|new|State.pm:DESTROY(949)|VCL::new process 12726 exiting
980 2010-04-01 15:45:08|5754|vcld:REAPER(744)|VCL process exited for reservation 10
981 2010-04-01 15:45:08|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:45:08
982 2010-04-01 15:45:13|5754|vcld:main(165)|lastcheckin time updated for management node 1: 2010-04-01 15:45:13
=====================================================================================
Thank you,
Amit
Re: VCL Getting Reservation to work
Posted by Ryan Johnson <rj...@gwmail.gwu.edu>.
I think some of the data that VCL needs is not initialized, could you post
your Image, ImageRevision, Computer, Vmprofile, and Vmhost tables?
On Thu, Apr 1, 2010 at 3:25 PM, Kumar, Amit H. <AH...@odu.edu> wrote:
> Hi All again,
>
> After a long break I am back to where, I was unable to get the Reservation
> to work:
>
> So, I was able to get Image capture work smoothly with the only short come
> of not completing the automatically created reservation process after image
> capture. Which failed at the point after registering the VM on the
> VMhost(ESXi). It does send the correct information and commands for
> registering the VM but it registeres the VM as "Unknown";
>
> Following upon Ryan's comment on un mounting and remounting the NFS store.
> I started to make reservations and yet haven't been successful.
>
> As usual something silly/unknown that I can't see.
> Two unusual things:
>
> 1) esx.pm:node_status(814)|could not set ssh identity variable for
> image type= host=
>
> 2) "SSH exits with the exit status of the remote command or with 255
> if an error occurred, output:" had same issue during image capture, and
> again Ryan pointed out a workaround to point to my NFS store directly.
> "Pushing the problem, doesn't not necessarily solve it" :)
>
> Please advise.......................................
>
> 3 2010-04-01 11:04:58|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:04:58
> 4 2010-04-01 11:05:03|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:03
> 5 2010-04-01 11:05:08|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:08
> 6 2010-04-01 11:05:13|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:13
> 7 2010-04-01 11:05:18|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:18
> 8 2010-04-01 11:05:23|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:23
> 9 2010-04-01 11:05:28|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:28
> 10 2010-04-01 11:05:33|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:33
> 11 2010-04-01 11:05:38|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:38
> 12 2010-04-01 11:05:43|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:43
> 13 2010-04-01 11:05:48|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:48
> 14 2010-04-01 11:05:53|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:53
> 15 2010-04-01 11:05:53|5754|8:8|new|utils.pm:reservation_being_processed(9634)|computerloadlog
> 'begin' entry does NOT exist for reservation 8
> 16 2010-04-01 11:05:53|5754|8:8|new|utils.pm:reservation_being_processed(9686)|reservation
> is NOT currently being processed
> 17 2010-04-01 11:05:53|5754|8:8|new|vcld:main(276)|reservation 8 is
> NOT already being processed
> 18
> 19 |5754|8:8|new| ---- WARNING ----
> 20 |5754|8:8|new| 2010-04-01 11:05:53|5754|8:8|new|utils.pm:get_request_info(5218)|preferredimageid
> is not set for computer id=2
> 21 |5754|8:8|new| ( 0) utils.pm, notify (line: 737)
> 22 |5754|8:8|new| (-1) utils.pm, get_request_info (line: 5218)
> 23 |5754|8:8|new| (-2) vcld, main (line: 280)
> 24
> 25 2010-04-01 11:05:53|5754|8:8|new|utils.pm:get_request_info(5354)|standalone
> affiliation found: Local
> 26 2010-04-01 11:05:53|5754|8:8|new|utils.pm:get_management_node_info(6815)|management
> node info retrieved from database for colbert
> 27 2010-04-01 11:05:53|5754|8:8|new|vcld:main(281)|retrieved request
> information from database
> 28 2010-04-01
> 11:05:53|5754|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 8
> 29 2010-04-01 11:05:53|5754|8:8|new|utils.pm:update_request_state(2186)|request
> 8 state updated to: pending, laststate to: new
> 30 2010-04-01 11:05:53|5754|8:8|new|utils.pm:insertloadlog(4710)|inserted
> computer=2, begin, beginning to process, state is new
> 31 2010-04-01 11:05:53|5754|8:8|new|vcld:make_new_child(509)|loaded
> VCL::new module
> 32 2010-04-01 11:05:53|12084|8:8|new|vcld:make_new_child(582)|vcld
> environment variable set to 0 for this process
> 33 2010-04-01 11:05:53|12084|8:8|new|Module.pm:new(132)|constructor
> called, class=VCL::new
> 34 2010-04-01 11:05:53|5754|8:8|new|vcld:make_new_child(566)|current
> number of forked kids: 1
> 35 2010-04-01 11:05:53|12084|8:8|new|Module.pm:new(154)|VCL::new
> object created
> 36 2010-04-01 11:05:53|12084|8:8|new|State.pm:initialize(85)|obtained
> a database handle for this state process, stored as $ENV{dbh}
> 37 2010-04-01 11:05:53|12084|8:8|new|State.pm:check_image_os(820)|no
> corrections need to be made, not an imaging request, returning 1
> 38 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 8
> 39 2010-04-01 11:05:53|12084|8:8|new|utils.pm:rename_vcld_process(7863)|reservation
> count: 1
> 40 2010-04-01 11:05:53|12084|8:8|new|utils.pm:rename_vcld_process(7882)|PARENTIMAGE:
> 1
> 41 2010-04-01 11:05:53|12084|8:8|new|utils.pm:rename_vcld_process(7883)|SUBIMAGE:
> 0
> 42 2010-04-01 11:05:53|12084|8:8|new|utils.pm:rename_vcld_process(7901)|renamed
> process to 'vcld VCL::new 8:8 new'
> 43 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 8
> 44 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 8
> 45 2010-04-01
> 11:05:53|12084|8:8|new|State.pm:initialize(121)|attempting to load
> provisioning module: VCL::Module::Provisioning::esx
> 46 2010-04-01 11:05:53|12084|8:8|new|esx.pm:initialize(138)|esx
> vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> 47 2010-04-01 11:05:53|12084|8:8|new|esx.pm:initialize(140)|vmware
> ESX module initialized
> 48 2010-04-01
> 11:05:53|12084|8:8|new|State.pm:initialize(127)|VCL::Module::Provisioning::esx
> module loaded
> 49 2010-04-01 11:05:53|12084|8:8|new|Module.pm:new(132)|constructor
> called, class=VCL::Module::Provisioning::esx
> 50 2010-04-01
> 11:05:53|12084|8:8|new|Module.pm:new(154)|VCL::Module::Provisioning::esx
> object created
> 51 2010-04-01 11:05:53|12084|8:8|new|esx.pm:initialize(138)|esx
> vmware toolkit root path found: /usr/lib/vmware-vcli/apps
> 52 2010-04-01 11:05:53|12084|8:8|new|esx.pm:initialize(140)|vmware
> ESX module initialized
> 53 2010-04-01
> 11:05:53|12084|8:8|new|State.pm:initialize(131)|VCL::Module::Provisioning::esx
> provisioner object created
> 54 2010-04-01
> 11:05:53|12084|8:8|new|State.pm:initialize(145)|attempting to load OS
> module: VCL::Module::OS::Linux
> 55 2010-04-01
> 11:05:53|12084|8:8|new|State.pm:initialize(151)|VCL::Module::OS::Linux
> module loaded
> 56 2010-04-01 11:05:53|12084|8:8|new|Module.pm:new(132)|constructor
> called, class=VCL::Module::OS::Linux
> 57 2010-04-01
> 11:05:53|12084|8:8|new|Module.pm:new(154)|VCL::Module::OS::Linux object
> created
> 58 2010-04-01
> 11:05:53|12084|8:8|new|State.pm:initialize(154)|VCL::Module::OS::Linux OS
> object created
> 59 2010-04-01
> 11:05:53|12084|8:8|new|State.pm:initialize(169)|returning 1
> 60 2010-04-01
> 11:05:53|12084|8:8|new|vcld:make_new_child(591)|VCL::new object created and
> initialized
> 61 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 8
> 62 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> to retrieve current state of computer esx35-hypervisor-1 from the database
> 63 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> current state of computer esx35-hypervisor-1 from the database: available
> 64 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(697)|data structure
> updated: $self->request_data->{reservation}{8}{computer}{state}{name}
> 65 |12084|8:8|new| computer_state_name = available
> 66
> 67 |12084|8:8|new| ---- WARNING ----
> 68 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data
> has not been initialized for get_computer_preferredimage_id:
> $self->request_data->{reservation}{8}{computer}{preferredimage}{id}
> 69 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 70 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
> 71 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 72 |12084|8:8|new| (-3) new.pm, process (line: 106)
> 73 |12084|8:8|new| (-4) vcld, make_new_child (line: 594)
> 74 |12084|8:8|new| (-5) vcld, main (line: 341)
> 75
> 76
> 77 |12084|8:8|new| ---- WARNING ----
> 78 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data
> has not been initialized for get_computer_preferredimage_name:
> $self->request_data->{reservation}{8}{computer}{preferredimage}{name}
> 79 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 80 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
> 81 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 82 |12084|8:8|new| (-3) new.pm, process (line: 107)
> 83 |12084|8:8|new| (-4) vcld, make_new_child (line: 594)
> 84 |12084|8:8|new| (-5) vcld, main (line: 341)
> 85
> 86 2010-04-01 11:05:53|12084|8:8|new|utils.pm:get_management_node_id(9364)|environment
> variable: 1
> 87 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(134)|reservation
> is parent = 1
> 88 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(135)|preload
> only = 0
> 89 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(136)|originating
> request state = new
> 90 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(137)|originating
> request laststate = new
> 91 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(138)|originating
> computer state = available
> 92 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(139)|originating
> computer type = blade
> 93 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 8
> 94 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> to retrieve current state of computer esx35-hypervisor-1 from the database
> 95 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> current state of computer esx35-hypervisor-1 from the database: available
> 96 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(697)|data structure
> updated: $self->request_data->{reservation}{8}{computer}{state}{name}
> 97 |12084|8:8|new| computer_state_name = available
> 98
> 99 |12084|8:8|new| ---- WARNING ----
> 100 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data
> has not been initialized for get_computer_preferredimage_id:
> $self->request_data->{reservation}{8}{computer}{preferredimage}{id}
> 101 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 102 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
> 103 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 104 |12084|8:8|new| (-3) new.pm, computer_not_being_used (line: 750)
> 105 |12084|8:8|new| (-4) new.pm, process (line: 170)
> 106 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
> 107 |12084|8:8|new| (-6) vcld, main (line: 341)
> 108
> 109
> 110 |12084|8:8|new| ---- WARNING ----
> 111 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data
> has not been initialized for get_computer_preferredimage_name:
> $self->request_data->{reservation}{8}{computer}{preferredimage}{name}
> 112 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 113 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
> 114 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 115 |12084|8:8|new| (-3) new.pm, computer_not_being_used (line: 751)
> 116 |12084|8:8|new| (-4) new.pm, process (line: 170)
> 117 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
> 118 |12084|8:8|new| (-6) vcld, main (line: 341)
> 119
> 120 2010-04-01 11:05:53|12084|8:8|new|utils.pm:get_management_node_id(9364)|environment
> variable: 1
> 121 2010-04-01 11:05:53|12084|8:8|new|new.pm:computer_not_being_used(788)|esx35-hypervisor-1
> state is available
> 122 2010-04-01 11:05:53|12084|8:8|new|new.pm:computer_not_being_used(798)|esx35-hypervisor-1
> is available, its state is available
> 123 2010-04-01 11:05:53|12084|8:8|new|new.pm:process(170)|esx35-hypervisor-1
> is not being used
> 124 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:is_parent_reservation(854)|returning
> true: parent reservation ID for this request: 8
> 125 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1946)|attempting
> to retrieve current state of computer esx35-hypervisor-1 from the database
> 126 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:get_computer_state_name(1977)|retrieved
> current state of computer esx35-hypervisor-1 from the database: available
> 127 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(697)|data structure
> updated: $self->request_data->{reservation}{8}{computer}{state}{name}
> 128 |12084|8:8|new| computer_state_name = available
> 129
> 130 |12084|8:8|new| ---- WARNING ----
> 131 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data
> has not been initialized for get_computer_preferredimage_id:
> $self->request_data->{reservation}{8}{computer}{preferredimage}{id}
> 132 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 133 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
> 134 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 135 |12084|8:8|new| (-3) new.pm, reload_image (line: 480)
> 136 |12084|8:8|new| (-4) new.pm, process (line: 266)
> 137 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
> 138 |12084|8:8|new| (-6) vcld, main (line: 341)
> 139
> 140
> 141 |12084|8:8|new| ---- WARNING ----
> 142 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data
> has not been initialized for get_computer_preferredimage_name:
> $self->request_data->{reservation}{8}{computer}{preferredimage}{name}
> 143 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 144 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
> 145 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 146 |12084|8:8|new| (-3) new.pm, reload_image (line: 481)
> 147 |12084|8:8|new| (-4) new.pm, process (line: 266)
> 148 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
> 149 |12084|8:8|new| (-6) vcld, main (line: 341)
> 150
> 151 2010-04-01 11:05:53|12084|8:8|new|utils.pm:get_management_node_id(9364)|environment
> variable: 1
> 152 2010-04-01 11:05:53|12084|8:8|new|new.pm:reload_image(514)|calling
> VCL::Module::Provisioning::esx->node_status()
> 153 2010-04-01 11:05:53|12084|8:8|new|utils.pm:insertloadlog(4710)|inserted
> computer=2, statuscheck, checking status of node
> 154
> 155 |12084|8:8|new| ---- WARNING ----
> 156 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data
> has not been initialized for get_vmhost_profile_vmpath:
> $self->request_data->{reservation}{8}{computer}{vmhost}{vmprofile}{vmpath}
> 157 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 158 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
> 159 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 160 |12084|8:8|new| (-3) esx.pm, node_status (line: 776)
> 161 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
> 162 |12084|8:8|new| (-5) new.pm, process (line: 266)
> 163 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
> 164
> 165
> 166 |12084|8:8|new| ---- WARNING ----
> 167 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data
> has not been initialized for get_vmhost_profile_datastore_path:
> $self->request_data->{reservation}{8}{computer}{vmhost}{vmprofile}{datastorepath}
> 168 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 169 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
> 170 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 171 |12084|8:8|new| (-3) esx.pm, node_status (line: 777)
> 172 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
> 173 |12084|8:8|new| (-5) new.pm, process (line: 266)
> 174 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
> 175
> 176
> 177 |12084|8:8|new| ---- WARNING ----
> 178 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data
> has not been initialized for get_vmhost_type:
> $self->request_data->{reservation}{8}{computer}{vmhost}{type}
> 179 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 180 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
> 181 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 182 |12084|8:8|new| (-3) esx.pm, node_status (line: 779)
> 183 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
> 184 |12084|8:8|new| (-5) new.pm, process (line: 266)
> 185 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
> 186
> 187
> 188 |12084|8:8|new| ---- WARNING ----
> 189 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data
> has not been initialized for get_vmhost_hostname:
> $self->request_data->{reservation}{8}{computer}{vmhost}{hostname}
> 190 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 191 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
> 192 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 193 |12084|8:8|new| (-3) esx.pm, node_status (line: 780)
> 194 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
> 195 |12084|8:8|new| (-5) new.pm, process (line: 266)
> 196 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
> 197
> 198
> 199 |12084|8:8|new| ---- WARNING ----
> 200 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|DataStructure.pm:_automethod(675)|corresponding data
> has not been initialized for get_vmhost_image_name:
> $self->request_data->{reservation}{8}{computer}{vmhost}{imagename}
> 201 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 202 |12084|8:8|new| (-1) DataStructure.pm, _automethod (line: 675)
> 203 |12084|8:8|new| (-2) Autoload.pm, __ANON__ (line: 80)
> 204 |12084|8:8|new| (-3) esx.pm, node_status (line: 781)
> 205 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
> 206 |12084|8:8|new| (-5) new.pm, process (line: 266)
> 207 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
> 208
> 216 2010-04-01 11:05:53|12084|8:8|new|esx.pm:node_status(787)|Entering
> node_status, checking status of esx35-hypervisor-1
> 217 2010-04-01 11:05:53|12084|8:8|new|esx.pm:node_status(788)|request_for_imaging:
> 0
> 218 2010-04-01 11:05:53|12084|8:8|new|esx.pm:node_status(789)|requeseted
> image name: esx3-basewinxp-v0
> 219 Use of uninitialized value in string eq at
> 220 /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/
> esx.pm line 804 (#1)
> 221 (W uninitialized) An undefined value was used as if it were
> already
> 222 defined. It was interpreted as a "" or a 0, but maybe it was a
> mistake.
> 223 To suppress this warning assign a defined value to your
> variables.
> 224
> 225 To help you figure out what was undefined, perl tells you what
> operation
> 226 you used the undefined value in. Note, however, that perl
> optimizes your
> 227 program and the operation displayed in the warning may not
> necessarily
> 228 appear literally in your program. For example, "that $foo" is
> 229 usually optimized into "that " . $foo, and the warning will
> refer to
> 230 the concatenation (.) operator, even though there is no . in
> your
> 231 program.
> 232
> 233
> 234 |12084|8:8|new| ---- WARNING ----
> 235 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|vcld:warning_handler(636)|Use of uninitialized value
> in string eq at /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pmline 804.
> 236 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 237 |12084|8:8|new| (-1) vcld, warning_handler (line: 636)
> 238 |12084|8:8|new| (-2) esx.pm, node_status (line: 804)
> 239 |12084|8:8|new| (-3) new.pm, reload_image (line: 518)
> 240 |12084|8:8|new| (-4) new.pm, process (line: 266)
> 241 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
> 242 |12084|8:8|new| (-6) vcld, main (line: 341)
> 243
> 244
> 245 |12084|8:8|new| ---- WARNING ----
> 246 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|vcld:warning_handler(636)|Use of uninitialized value
> in pattern match (m//) at /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/
> esx.pm line 811.
> 247 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 248 |12084|8:8|new| (-1) vcld, warning_handler (line: 636)
> 249 |12084|8:8|new| (-2) esx.pm, node_status (line: 811)
> 250 |12084|8:8|new| (-3) new.pm, reload_image (line: 518)
> 251 |12084|8:8|new| (-4) new.pm, process (line: 266)
> 252 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
> 253 |12084|8:8|new| (-6) vcld, main (line: 341)
> 254
> 255
> 256 |12084|8:8|new| ---- WARNING ----
> 257 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|vcld:warning_handler(636)|Use of uninitialized value
> in concatenation (.) or string at
> /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 815.
> 258 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 259 |12084|8:8|new| (-1) vcld, warning_handler (line: 636)
> 260 |12084|8:8|new| (-2) esx.pm, node_status (line: 814)
> 261 |12084|8:8|new| (-3) new.pm, reload_image (line: 518)
> 262 |12084|8:8|new| (-4) new.pm, process (line: 266)
> 263 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
> 264 |12084|8:8|new| (-6) vcld, main (line: 341)
> 265
> 266
> 267 |12084|8:8|new| ---- WARNING ----
> 268 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|vcld:warning_handler(636)|Use of uninitialized value
> in concatenation (.) or string at
> /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 815.
> 269 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 270 |12084|8:8|new| (-1) vcld, warning_handler (line: 636)
> 271 |12084|8:8|new| (-2) esx.pm, node_status (line: 814)
> 272 |12084|8:8|new| (-3) new.pm, reload_image (line: 518)
> 273 |12084|8:8|new| (-4) new.pm, process (line: 266)
> 274 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
> 275 |12084|8:8|new| (-6) vcld, main (line: 341)
> 276
> 277
> 278 |12084|8:8|new| ---- WARNING ----
> 279 |12084|8:8|new| 2010-04-01
> 11:05:53|12084|8:8|new|vcld:warning_handler(636)|Use of uninitialized value
> in concatenation (.) or string at
> /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 815.
> 280 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 281 |12084|8:8|new| (-1) vcld, warning_handler (line: 636)
> 282 |12084|8:8|new| (-2) esx.pm, node_status (line: 814)
> 283 |12084|8:8|new| (-3) new.pm, reload_image (line: 518)
> 284 |12084|8:8|new| (-4) new.pm, process (line: 266)
> 285 |12084|8:8|new| (-5) vcld, make_new_child (line: 594)
> 286 |12084|8:8|new| (-6) vcld, main (line: 341)
> 287
> 288 No recipient addresses found in header
> 289 2010-04-01 11:05:54|12084|8:8|new|utils.pm:mail(1348)|SUCCESS --
> Sending mail To: , PROBLEM -- esx.pm
> 290
> 291 |12084|8:8|new| ---- CRITICAL ----
> 292 |12084|8:8|new| 2010-04-01 11:05:53|12084|8:8|new|esx.pm:node_status(814)|could
> not set ssh identity variable for image type= host=
> 293 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 294 |12084|8:8|new| (-1) esx.pm, node_status (line: 814)
> 295 |12084|8:8|new| (-2) new.pm, reload_image (line: 518)
> 296 |12084|8:8|new| (-3) new.pm, process (line: 266)
> 297 |12084|8:8|new| (-4) vcld, make_new_child (line: 594)
> 298 |12084|8:8|new| (-5) vcld, main (line: 341)
> 299
> 300 2010-04-01 11:05:54|12084|8:8|new|esx.pm:node_status(819)|checking
> if esx35-hypervisor-1 is pingable
> 301 2010-04-01 11:05:54|12084|8:8|new|esx.pm:node_status(822)|esx35-hypervisor-1
> is pingable (1)
> 302 2010-04-01 11:05:54|12084|8:8|new|esx.pm:node_status(836)|Trying
> to ssh...
> 303 2010-04-01 11:05:54|12084|8:8|new|utils.pm:run_ssh_command(6180)|executing
> SSH command on esx35-hypervisor-1:
> 304 |12084|8:8|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> esx35-hypervisor-1 'uname -s' 2>&1
> 305
> 306 |12084|8:8|new| ---- WARNING ----
> 307 |12084|8:8|new| 2010-04-01 11:05:54|12084|8:8|new|utils.pm:run_ssh_command(6252)|attempt
> 1/3: failed to execute SSH command on esx35-hypervisor-1: uname -s, exit
> status: 255, SSH exits with the exit status of the remote command or with
> 255 if an error occurred, output:
> 308 |12084|8:8|new| ssh output (uname -s): VMkernel
> 309 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 310 |12084|8:8|new| (-1) utils.pm, run_ssh_command (line: 6252)
> 311 |12084|8:8|new| (-2) utils.pm, _sshd_status (line: 2755)
> 312 |12084|8:8|new| (-3) esx.pm, node_status (line: 839)
> 313 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
> 314 |12084|8:8|new| (-5) new.pm, process (line: 266)
> 315 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
> 316
> 317 2010-04-01 11:05:54|12084|8:8|new|utils.pm:run_ssh_command(6172)|sleeping
> for 2 seconds before making next SSH attempt
> 318 2010-04-01 11:05:56|12084|8:8|new|utils.pm:run_ssh_command(6184)|attempt
> 2/3: executing SSH command on esx35-hypervisor-1:
> 319 |12084|8:8|new| /usr/bin/ssh -i /etc/vcl/vcl.key -l root -p 22 -x
> esx35-hypervisor-1 'uname -s' 2>&1
> 320
> 321 |12084|8:8|new| ---- WARNING ----
> 322 |12084|8:8|new| 2010-04-01 11:05:56|12084|8:8|new|utils.pm:run_ssh_command(6252)|attempt
> 2/3: failed to execute SSH command on esx35-hypervisor-1: uname -s, exit
> status: 255, SSH exits with the exit status of the remote command or with
> 255 if an error occurred, output:
> 323 |12084|8:8|new| ssh output (uname -s): VMkernel
> 324 |12084|8:8|new| ( 0) utils.pm, notify (line: 737)
> 325 |12084|8:8|new| (-1) utils.pm, run_ssh_command (line: 6252)
> 326 |12084|8:8|new| (-2) utils.pm, _sshd_status (line: 2755)
> 327 |12084|8:8|new| (-3) esx.pm, node_status (line: 839)
> 328 |12084|8:8|new| (-4) new.pm, reload_image (line: 518)
> 329 |12084|8:8|new| (-5) new.pm, process (line: 266)
> 330 |12084|8:8|new| (-6) vcld, make_new_child (line: 594)
> 331
> 332 2010-04-01 11:05:56|12084|8:8|new|utils.pm:run_ssh_command(6172)|sleeping
> for 15 seconds before making next SSH attempt
> 333 2010-04-01 11:05:58|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:05:58
> 334 2010-04-01 11:06:03|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:06:03
> 335 2010-04-01 11:06:08|5754|vcld:main(165)|lastcheckin time updated
> for management node 1: 2010-04-01 11:06:08
> 336
>
>
> Thank you,
> Amit
>
>