You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by "Richard Klein (RSI)" <rk...@rsitex.com> on 2016/04/02 21:04:44 UTC
Cloudstack 4.7 password reset issue.
First time mailer here, hello everyone. We are implementing CS 4.7 cluster with CentOS7. All is going well with a few bumps that we have resolved. During testing we are getting an unusual behavior from the password reset process. We have setup the templates using the instructions from the documentation including installing the guest password and SSH setup scripts. Below is what happens:
* Create a VM from template that has the password option checked and the root password is set as expected.
* Log in with 1st password and then change password of root via the console of the VM
* Perform a reboot from the VM console and the password stays as set manually.
* Power off VM, change PW with CS web GUI. When the VM is powered on the password is the new one (as expected).
* Log in with new password on the console and then manually change root again.
* Reboot from the console and the password remains as manually set.
* Power off VM via web GUI and then power it back on.
* The password reverts to the last password reset operation from the web GUI.
I've looked at the message logs and the guest password and SSH scripts are running normally. The virtual router is sending the password reset again when the service query it. But this only occurs after the VM is powered off then on again. I expected the password to remain as the manually set password but it always reverts to the last password reset operation when the VM is powered off then on again.
Any insight would be helpful.
Richard Klein <rk...@rsitex.com>
RSI
5426 Guadalupe, Suite 100
Austin TX 78751
Re: Cloudstack 4.7 password reset issue.
Posted by Rajani Karuturi <ra...@apache.org>.
Can you check the vm_instance table for the VR entry and update
vm_template_id?
This might be helpful https://gist.github.com/terbolous/102ae8edd1cda192561c
~Rajani
On Sat, Apr 9, 2016 at 5:45 AM, Richard Klein (RSI) <rk...@rsitex.com>
wrote:
> I found the password reset issue and it ended up being a Python script on
> the VR. I ended up modifying the "/opt/cloud/bin/configure.py" to resolve
> the issue. Basically there is a "/etc/cloud/vmpassword.json" file that is
> updated with the IP/password pair when the GUI password change is
> performed. During the power on process the VM configuration info is sent
> to the router which reads the vmpassword.json file and sends the password
> changes to the password server cache file. When the client retrieved the
> password it was cleared from the password cache file but not the
> vmpassword.json file. So every time a VM started the last password reset
> was sent to the password server again.
>
> The question I have now is how do I get the system VM template updated
> with the change? Since we are using CS v4.7 we used the system template
> for v4.6 per the installation instructions for CentOS7 and KVM. I
> performed the following steps to use a new system VM template:
>
> * I copied the system vm template QCOW2 file from secondary storage to a
> work server and made a backup of it.
> * On the work server I mounted the QCOW2 template file using "guestmount"
> tools and made the code changes to the template.
> * I then copied this modified template file to a web server and registered
> the template in cloudstack with all checkboxes off except for "routing".
> * Then we set the cloudstack global value of "router.template.kvm" to the
> name of the new template.
> * The management services were restarted.
> * I picked a test VR, powered it off, destroyed it then let the system
> recreate it.
> * When I look at the code I changed on the new VR it does not appear.
>
> I even doubled checked the database and the vm_instance table for the test
> VR showed the new template ID. I must be missing something or I don't
> really understand how the system templates are created. Any
> help/suggestions would be appreciated.
>
>
>
> Richard Klein <rk...@rsitex.com>
> RSI
> 5426 Guadalupe, Suite 100
> Austin TX 78751
>
>
>
> > -----Original Message-----
> > From: Richard Klein (RSI)
> > Sent: Tuesday, April 05, 2016 2:32 PM
> > To: users@cloudstack.apache.org
> > Subject: RE: Cloudstack 4.7 password reset issue.
> >
> > The snippets for before and after the reboot via console look the same
> so I
> > pasted the 2nd set of message instead of the first. Sorry about that.
> I did
> > discover that the /var/lib/dhclient/dhclient.leases existed but was
> empty. I've
> > run across an issue with CentOS 7 where the lease file is missing so I
> wrote a
> > "cloud-dhcp-check" service that makes sure it exists but now I need to
> validate
> > its content. That being said, I have insured that the dhclient_leases
> was valid
> > and replicated the problem.
> >
> > The cloud-set-guest-xxxx scripts are from the master branch GitHub
> repository
> > for apaches/cloudstack using the
> > "
> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-set-
> > guest-password.in" and the
> > "
> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-set-
> > guest-sshkey.in" links.
> >
> > I have attached the entire log from the VR but have some snippets below
> along
> > with the VM client logs and the issue still occurs after fixing the dhcp
> lease file.
> > I did not perform any password resets via the GUI during this process.
> >
>
>
RE: Cloudstack 4.7 password reset issue.
Posted by "Richard Klein (RSI)" <rk...@rsitex.com>.
Thanks for the explanation, which really clarified a lot. I did as you suggested and changed the code in a local development WS and then rebuilt the RPMs using the following steps:
* Updated code in a local git repository.
* Ran the " mvn clean install -P systemvm ".
* Ran the "./package.sh -d centos7" to create the new RPM's
* Copied the new RPMs to a local repository we have to control updates.
* Stopped the cloudstack-management, cloudstack-usage and cloudstack-agent services
* Reinstalled the "cloudstack-common" on all management and hosts using the "yum reinstall cloudstack-common".
* Reinstalled the "cloudstack-agent" on the hosts only using "yum reinstall cloudstack-agent"
* Started the agents on all hosts.
* Started management services.
* Stopped and then started a test VR but get the following error: Resource [Host:31] is unreachable: Host 31: Unable to start instance due to Unable to start VM[DomainRouter|r-1187-VM] due to error in finalizeStart, not retrying
I also restored the system vm template on secondary storage from a backup I made before starting any changes. I tried 2 different VRs and I get this same message with a different Host:xx value.
I reverted back to the original code by repeating the above process using the original RPMs. The VR start normally but I still have the original password reset issue. Ether I am missing something with the upgrade to a new VM system template process or I guess the code change could have caused this problem. I did test the code change by using a live VR and manually make the change. All worked well then but I can't test the stop/restart process cause it reverts back to the original scripts.
Did I miss a step or do something incorrectly with the new system VM update? Thanks.
----------------------------------------------
Richard Klein <rk...@rsitex.com>
RSI
5426 Guadalupe, Suite 100
Austin TX 78751
> -----Original Message-----
> From: Remi Bergsma [mailto:RBergsma@schubergphilis.com]
> Sent: Saturday, April 09, 2016 9:48 AM
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack 4.7 password reset issue.
>
> Hi,
>
> It’s best to make a change in the source of CloudStack, then send it as a Pull
> Request to get it included.
>
> When you run: `mvn clean install -P systemvm` a new systemvm.iso is
> generated.
>
> However, the management server will check the systemvm.iso’s signature, so
> I’m not sure if replacing works without also copying it to the management
> server and restart it (so it generates a new signature).
>
> Another trick is to alter the scripts on live routers with Ansible.
>
> Regards,
> Remi
>
>
>
>
> On 09/04/16 03:42, "Simon Weller" <sw...@ena.com> wrote:
>
> >
> >Richard,
> >
> >The system vms are bootstrapped with an iso that is mounted and all the
> scripts are installed from there. The reason for this is to reduce the number of
> times you have to completely redeploy routers (we used to have to do it for
> every release).
> >My guess is the file you need to change is within that iso.
> >The iso is part of the cloudstack-common package that is installed onto all the
> hosts (along with the cloudstack-agent).
> >Take a look in /usr/share/cloudstack-common/vms. There should be an iso
> >in there called systemvm.iso
> >
> >You'll probably need to mount the iso make your changes to the filesystem,
> rebuild the iso with mkisofs, copy that new iso to all of your hosts and then stop
> and start your VR.
> >
> >Hope this helps.
> >
> >- Si
> >
> >
> >
> >
> >________________________________________
> >From: Richard Klein (RSI) <rk...@rsitex.com>
> >Sent: Friday, April 8, 2016 7:15 PM
> >To: users@cloudstack.apache.org
> >Subject: RE: Cloudstack 4.7 password reset issue.
> >
> >I found the password reset issue and it ended up being a Python script on the
> VR. I ended up modifying the "/opt/cloud/bin/configure.py" to resolve the
> issue. Basically there is a "/etc/cloud/vmpassword.json" file that is updated
> with the IP/password pair when the GUI password change is performed. During
> the power on process the VM configuration info is sent to the router which
> reads the vmpassword.json file and sends the password changes to the
> password server cache file. When the client retrieved the password it was
> cleared from the password cache file but not the vmpassword.json file. So
> every time a VM started the last password reset was sent to the password
> server again.
> >
> >The question I have now is how do I get the system VM template updated with
> the change? Since we are using CS v4.7 we used the system template for v4.6
> per the installation instructions for CentOS7 and KVM. I performed the
> following steps to use a new system VM template:
> >
> >* I copied the system vm template QCOW2 file from secondary storage to a
> work server and made a backup of it.
> >* On the work server I mounted the QCOW2 template file using "guestmount"
> tools and made the code changes to the template.
> >* I then copied this modified template file to a web server and registered the
> template in cloudstack with all checkboxes off except for "routing".
> >* Then we set the cloudstack global value of "router.template.kvm" to the
> name of the new template.
> >* The management services were restarted.
> >* I picked a test VR, powered it off, destroyed it then let the system recreate
> it.
> >* When I look at the code I changed on the new VR it does not appear.
> >
> >I even doubled checked the database and the vm_instance table for the test
> VR showed the new template ID. I must be missing something or I don't really
> understand how the system templates are created. Any help/suggestions would
> be appreciated.
> >
> >
> >
> >Richard Klein <rk...@rsitex.com>
> >RSI
> >5426 Guadalupe, Suite 100
> >Austin TX 78751
> >
> >
> >
> >> -----Original Message-----
> >> From: Richard Klein (RSI)
> >> Sent: Tuesday, April 05, 2016 2:32 PM
> >> To: users@cloudstack.apache.org
> >> Subject: RE: Cloudstack 4.7 password reset issue.
> >>
> >> The snippets for before and after the reboot via console look the
> >> same so I pasted the 2nd set of message instead of the first. Sorry
> >> about that. I did discover that the
> >> /var/lib/dhclient/dhclient.leases existed but was empty. I've run
> >> across an issue with CentOS 7 where the lease file is missing so I
> >> wrote a "cloud-dhcp-check" service that makes sure it exists but now
> >> I need to validate its content. That being said, I have insured that the
> dhclient_leases was valid and replicated the problem.
> >>
> >> The cloud-set-guest-xxxx scripts are from the master branch GitHub
> >> repository for apaches/cloudstack using the
> >> "https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-
> >> set-
> >> guest-password.in" and the
> >> "https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-
> >> set-
> >> guest-sshkey.in" links.
> >>
> >> I have attached the entire log from the VR but have some snippets
> >> below along with the VM client logs and the issue still occurs after fixing the
> dhcp lease file.
> >> I did not perform any password resets via the GUI during this process.
> >>
> >
Re: Cloudstack 4.7 password reset issue.
Posted by Remi Bergsma <RB...@schubergphilis.com>.
Hi,
It’s best to make a change in the source of CloudStack, then send it as a Pull Request to get it included.
When you run: `mvn clean install -P systemvm` a new systemvm.iso is generated.
However, the management server will check the systemvm.iso’s signature, so I’m not sure if replacing works without also copying it to the management server and restart it (so it generates a new signature).
Another trick is to alter the scripts on live routers with Ansible.
Regards,
Remi
On 09/04/16 03:42, "Simon Weller" <sw...@ena.com> wrote:
>
>Richard,
>
>The system vms are bootstrapped with an iso that is mounted and all the scripts are installed from there. The reason for this is to reduce the number of times you have to completely redeploy routers (we used to have to do it for every release).
>My guess is the file you need to change is within that iso.
>The iso is part of the cloudstack-common package that is installed onto all the hosts (along with the cloudstack-agent).
>Take a look in /usr/share/cloudstack-common/vms. There should be an iso in there called systemvm.iso
>
>You'll probably need to mount the iso make your changes to the filesystem, rebuild the iso with mkisofs, copy that new iso to all of your hosts and then stop and start your VR.
>
>Hope this helps.
>
>- Si
>
>
>
>
>________________________________________
>From: Richard Klein (RSI) <rk...@rsitex.com>
>Sent: Friday, April 8, 2016 7:15 PM
>To: users@cloudstack.apache.org
>Subject: RE: Cloudstack 4.7 password reset issue.
>
>I found the password reset issue and it ended up being a Python script on the VR. I ended up modifying the "/opt/cloud/bin/configure.py" to resolve the issue. Basically there is a "/etc/cloud/vmpassword.json" file that is updated with the IP/password pair when the GUI password change is performed. During the power on process the VM configuration info is sent to the router which reads the vmpassword.json file and sends the password changes to the password server cache file. When the client retrieved the password it was cleared from the password cache file but not the vmpassword.json file. So every time a VM started the last password reset was sent to the password server again.
>
>The question I have now is how do I get the system VM template updated with the change? Since we are using CS v4.7 we used the system template for v4.6 per the installation instructions for CentOS7 and KVM. I performed the following steps to use a new system VM template:
>
>* I copied the system vm template QCOW2 file from secondary storage to a work server and made a backup of it.
>* On the work server I mounted the QCOW2 template file using "guestmount" tools and made the code changes to the template.
>* I then copied this modified template file to a web server and registered the template in cloudstack with all checkboxes off except for "routing".
>* Then we set the cloudstack global value of "router.template.kvm" to the name of the new template.
>* The management services were restarted.
>* I picked a test VR, powered it off, destroyed it then let the system recreate it.
>* When I look at the code I changed on the new VR it does not appear.
>
>I even doubled checked the database and the vm_instance table for the test VR showed the new template ID. I must be missing something or I don't really understand how the system templates are created. Any help/suggestions would be appreciated.
>
>
>
>Richard Klein <rk...@rsitex.com>
>RSI
>5426 Guadalupe, Suite 100
>Austin TX 78751
>
>
>
>> -----Original Message-----
>> From: Richard Klein (RSI)
>> Sent: Tuesday, April 05, 2016 2:32 PM
>> To: users@cloudstack.apache.org
>> Subject: RE: Cloudstack 4.7 password reset issue.
>>
>> The snippets for before and after the reboot via console look the same so I
>> pasted the 2nd set of message instead of the first. Sorry about that. I did
>> discover that the /var/lib/dhclient/dhclient.leases existed but was empty. I've
>> run across an issue with CentOS 7 where the lease file is missing so I wrote a
>> "cloud-dhcp-check" service that makes sure it exists but now I need to validate
>> its content. That being said, I have insured that the dhclient_leases was valid
>> and replicated the problem.
>>
>> The cloud-set-guest-xxxx scripts are from the master branch GitHub repository
>> for apaches/cloudstack using the
>> "https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-set-
>> guest-password.in" and the
>> "https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-set-
>> guest-sshkey.in" links.
>>
>> I have attached the entire log from the VR but have some snippets below along
>> with the VM client logs and the issue still occurs after fixing the dhcp lease file.
>> I did not perform any password resets via the GUI during this process.
>>
>
Re: Cloudstack 4.7 password reset issue.
Posted by Simon Weller <sw...@ena.com>.
Richard,
The system vms are bootstrapped with an iso that is mounted and all the scripts are installed from there. The reason for this is to reduce the number of times you have to completely redeploy routers (we used to have to do it for every release).
My guess is the file you need to change is within that iso.
The iso is part of the cloudstack-common package that is installed onto all the hosts (along with the cloudstack-agent).
Take a look in /usr/share/cloudstack-common/vms. There should be an iso in there called systemvm.iso
You'll probably need to mount the iso make your changes to the filesystem, rebuild the iso with mkisofs, copy that new iso to all of your hosts and then stop and start your VR.
Hope this helps.
- Si
________________________________________
From: Richard Klein (RSI) <rk...@rsitex.com>
Sent: Friday, April 8, 2016 7:15 PM
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.7 password reset issue.
I found the password reset issue and it ended up being a Python script on the VR. I ended up modifying the "/opt/cloud/bin/configure.py" to resolve the issue. Basically there is a "/etc/cloud/vmpassword.json" file that is updated with the IP/password pair when the GUI password change is performed. During the power on process the VM configuration info is sent to the router which reads the vmpassword.json file and sends the password changes to the password server cache file. When the client retrieved the password it was cleared from the password cache file but not the vmpassword.json file. So every time a VM started the last password reset was sent to the password server again.
The question I have now is how do I get the system VM template updated with the change? Since we are using CS v4.7 we used the system template for v4.6 per the installation instructions for CentOS7 and KVM. I performed the following steps to use a new system VM template:
* I copied the system vm template QCOW2 file from secondary storage to a work server and made a backup of it.
* On the work server I mounted the QCOW2 template file using "guestmount" tools and made the code changes to the template.
* I then copied this modified template file to a web server and registered the template in cloudstack with all checkboxes off except for "routing".
* Then we set the cloudstack global value of "router.template.kvm" to the name of the new template.
* The management services were restarted.
* I picked a test VR, powered it off, destroyed it then let the system recreate it.
* When I look at the code I changed on the new VR it does not appear.
I even doubled checked the database and the vm_instance table for the test VR showed the new template ID. I must be missing something or I don't really understand how the system templates are created. Any help/suggestions would be appreciated.
Richard Klein <rk...@rsitex.com>
RSI
5426 Guadalupe, Suite 100
Austin TX 78751
> -----Original Message-----
> From: Richard Klein (RSI)
> Sent: Tuesday, April 05, 2016 2:32 PM
> To: users@cloudstack.apache.org
> Subject: RE: Cloudstack 4.7 password reset issue.
>
> The snippets for before and after the reboot via console look the same so I
> pasted the 2nd set of message instead of the first. Sorry about that. I did
> discover that the /var/lib/dhclient/dhclient.leases existed but was empty. I've
> run across an issue with CentOS 7 where the lease file is missing so I wrote a
> "cloud-dhcp-check" service that makes sure it exists but now I need to validate
> its content. That being said, I have insured that the dhclient_leases was valid
> and replicated the problem.
>
> The cloud-set-guest-xxxx scripts are from the master branch GitHub repository
> for apaches/cloudstack using the
> "https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-set-
> guest-password.in" and the
> "https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-set-
> guest-sshkey.in" links.
>
> I have attached the entire log from the VR but have some snippets below along
> with the VM client logs and the issue still occurs after fixing the dhcp lease file.
> I did not perform any password resets via the GUI during this process.
>
RE: Cloudstack 4.7 password reset issue.
Posted by "Richard Klein (RSI)" <rk...@rsitex.com>.
I found the password reset issue and it ended up being a Python script on the VR. I ended up modifying the "/opt/cloud/bin/configure.py" to resolve the issue. Basically there is a "/etc/cloud/vmpassword.json" file that is updated with the IP/password pair when the GUI password change is performed. During the power on process the VM configuration info is sent to the router which reads the vmpassword.json file and sends the password changes to the password server cache file. When the client retrieved the password it was cleared from the password cache file but not the vmpassword.json file. So every time a VM started the last password reset was sent to the password server again.
The question I have now is how do I get the system VM template updated with the change? Since we are using CS v4.7 we used the system template for v4.6 per the installation instructions for CentOS7 and KVM. I performed the following steps to use a new system VM template:
* I copied the system vm template QCOW2 file from secondary storage to a work server and made a backup of it.
* On the work server I mounted the QCOW2 template file using "guestmount" tools and made the code changes to the template.
* I then copied this modified template file to a web server and registered the template in cloudstack with all checkboxes off except for "routing".
* Then we set the cloudstack global value of "router.template.kvm" to the name of the new template.
* The management services were restarted.
* I picked a test VR, powered it off, destroyed it then let the system recreate it.
* When I look at the code I changed on the new VR it does not appear.
I even doubled checked the database and the vm_instance table for the test VR showed the new template ID. I must be missing something or I don't really understand how the system templates are created. Any help/suggestions would be appreciated.
Richard Klein <rk...@rsitex.com>
RSI
5426 Guadalupe, Suite 100
Austin TX 78751
> -----Original Message-----
> From: Richard Klein (RSI)
> Sent: Tuesday, April 05, 2016 2:32 PM
> To: users@cloudstack.apache.org
> Subject: RE: Cloudstack 4.7 password reset issue.
>
> The snippets for before and after the reboot via console look the same so I
> pasted the 2nd set of message instead of the first. Sorry about that. I did
> discover that the /var/lib/dhclient/dhclient.leases existed but was empty. I've
> run across an issue with CentOS 7 where the lease file is missing so I wrote a
> "cloud-dhcp-check" service that makes sure it exists but now I need to validate
> its content. That being said, I have insured that the dhclient_leases was valid
> and replicated the problem.
>
> The cloud-set-guest-xxxx scripts are from the master branch GitHub repository
> for apaches/cloudstack using the
> "https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-set-
> guest-password.in" and the
> "https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-set-
> guest-sshkey.in" links.
>
> I have attached the entire log from the VR but have some snippets below along
> with the VM client logs and the issue still occurs after fixing the dhcp lease file.
> I did not perform any password resets via the GUI during this process.
>
RE: Cloudstack 4.7 password reset issue.
Posted by "Richard Klein (RSI)" <rk...@rsitex.com>.
The snippets for before and after the reboot via console look the same so I pasted the 2nd set of message instead of the first. Sorry about that. I did discover that the /var/lib/dhclient/dhclient.leases existed but was empty. I've run across an issue with CentOS 7 where the lease file is missing so I wrote a "cloud-dhcp-check" service that makes sure it exists but now I need to validate its content. That being said, I have insured that the dhclient_leases was valid and replicated the problem.
The cloud-set-guest-xxxx scripts are from the master branch GitHub repository for apaches/cloudstack using the "https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-set-guest-password.in" and the "https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-set-guest-sshkey.in" links.
I have attached the entire log from the VR but have some snippets below along with the VM client logs and the issue still occurs after fixing the dhcp lease file. I did not perform any password resets via the GUI during this process.
== Logs from local link on the corresponding VR for April 5th only (time is UTC) ==
grep "CsHelper.*10\.1\.1\.39" /var/log/cloud.log
2016-04-05 18:32:41,103 CsHelper.py execute:160 Executing: curl --header "DomU_Request: save_password" "http://10.1.1.1:8080/" -F "ip=10.1.1.39" -F "password=a8Q7Bs" -F "token=ae199529efcd26f7811f36ae7c0807b1" >/dev/null 2>/dev/null &
2016-04-05 18:32:44,806 CsHelper.py execute:160 Executing: curl --header "DomU_Request: save_password" "http://10.1.1.1:8080/" -F "ip=10.1.1.39" -F "password=a8Q7Bs" -F "token=ae199529efcd26f7811f36ae7c0807b1" >/dev/null 2>/dev/null &
2016-04-05 18:38:21,043 CsHelper.py execute:160 Executing: curl --header "DomU_Request: save_password" "http://10.1.1.1:8080/" -F "ip=10.1.1.39" -F "password=a8Q7Bs" -F "token=ae199529efcd26f7811f36ae7c0807b1" >/dev/null 2>/dev/null &
2016-04-05 18:38:24,454 CsHelper.py execute:160 Executing: curl --header "DomU_Request: save_password" "http://10.1.1.1:8080/" -F "ip=10.1.1.39" -F "password=a8Q7Bs" -F "token=ae199529efcd26f7811f36ae7c0807b1" >/dev/null 2>/dev/null &
== Logs from the VM client using "grep cloud /var/log/messages" (time is CDT) ==
-- Power off via GUI --
Apr 5 13:29:57 rsi-test-c7 systemd: Stopped Execute cloud user/final scripts.
Apr 5 13:29:57 rsi-test-c7 systemd: Stopping Execute cloud user/final scripts...
Apr 5 13:29:57 rsi-test-c7 systemd: Stopped Apply the settings specified in cloud-config.
Apr 5 13:29:57 rsi-test-c7 systemd: Stopping Apply the settings specified in cloud-config...
-- Powered on via GUI --
Apr 5 13:33:07 rsi-test-c7 systemd: Starting Initial cloud-init job (pre-networking)...
Apr 5 13:33:09 rsi-test-c7 systemd: Started Initial cloud-init job (pre-networking).
Apr 5 13:33:11 rsi-test-c7 NetworkManager[539]: <info> domain name 'cs2cloud.internal'
Apr 5 13:33:17 rsi-test-c7 systemd: Starting LSB: cloud-dhcp-check...
Apr 5 13:33:17 rsi-test-c7 cloud-dhcp-check: Checking for DHCP leas file...
Apr 5 13:33:17 rsi-test-c7 cloud-dhcp-check: DHCP lease found: /var/lib/dhclient/dhclient.leases
Apr 5 13:33:17 rsi-test-c7 systemd: Started LSB: cloud-dhcp-check.
Apr 5 13:33:17 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-password...
Apr 5 13:33:17 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-sshkey...
Apr 5 13:33:17 rsi-test-c7 cloud: Found ssh key server at 10.1.1.1
Apr 5 13:33:17 rsi-test-c7 cloud: Found password server IP 10.1.1.1 in /var/lib/dhclient/dhclient.leases
Apr 5 13:33:17 rsi-test-c7 cloud: Sending request to password server at 10.1.1.1
Apr 5 13:33:17 rsi-test-c7 cloud: Sending request to ssh key server at 10.1.1.1
Apr 5 13:33:17 rsi-test-c7 cloud: Got response from server at 10.1.1.1
Apr 5 13:33:17 rsi-test-c7 cloud: Did not receive any keys from any server
Apr 5 13:33:17 rsi-test-c7 systemd: cloud-set-guest-sshkey.service: control process exited, code=exited status=1
Apr 5 13:33:17 rsi-test-c7 systemd: Failed to start LSB: cloud-set-guest-sshkey.
Apr 5 13:33:17 rsi-test-c7 systemd: Unit cloud-set-guest-sshkey.service entered failed state.
Apr 5 13:33:17 rsi-test-c7 systemd: cloud-set-guest-sshkey.service failed.
Apr 5 13:33:17 rsi-test-c7 cloud: Got response from server at 10.1.1.1
Apr 5 13:33:17 rsi-test-c7 cloud: VM got a valid password from server at 10.1.1.1
Apr 5 13:33:17 rsi-test-c7 cloud: Changing password for user root
Apr 5 13:33:17 rsi-test-c7 cloud: Sending acknowledgment to password server at 10.1.1.1
Apr 5 13:33:17 rsi-test-c7 systemd: Started LSB: cloud-set-guest-password.
Apr 5 13:33:17 rsi-test-c7 cloud-set-guest-password: saved_password
Apr 5 13:33:17 rsi-test-c7 systemd: Starting Initial cloud-init job (metadata service crawler)...
Apr 5 13:33:18 rsi-test-c7 systemd: Started Initial cloud-init job (metadata service crawler).
Apr 5 13:33:18 rsi-test-c7 systemd: Starting Apply the settings specified in cloud-config...
Apr 5 13:33:19 rsi-test-c7 systemd: Started Apply the settings specified in cloud-config.
Apr 5 13:33:19 rsi-test-c7 systemd: Starting Execute cloud user/final scripts...
Apr 5 13:33:19 rsi-test-c7 systemd: Started Execute cloud user/final scripts.
-- Changed root password via VM console then powered off via GUI --
Apr 5 13:37:10 rsi-test-c7 systemd: Stopped Execute cloud user/final scripts.
Apr 5 13:37:10 rsi-test-c7 systemd: Stopping Execute cloud user/final scripts...
Apr 5 13:37:10 rsi-test-c7 systemd: Stopped Apply the settings specified in cloud-config.
Apr 5 13:37:10 rsi-test-c7 systemd: Stopping Apply the settings specified in cloud-config...
-- Powered on via GUI --
Apr 5 13:38:47 rsi-test-c7 systemd: Starting Initial cloud-init job (pre-networking)...
Apr 5 13:38:49 rsi-test-c7 systemd: Started Initial cloud-init job (pre-networking).
Apr 5 13:38:51 rsi-test-c7 NetworkManager[535]: <info> domain name 'cs2cloud.internal'
Apr 5 13:38:56 rsi-test-c7 systemd: Starting LSB: cloud-dhcp-check...
Apr 5 13:38:56 rsi-test-c7 cloud-dhcp-check: Checking for DHCP leas file...
Apr 5 13:38:57 rsi-test-c7 cloud-dhcp-check: DHCP lease found: /var/lib/dhclient/dhclient.leases
Apr 5 13:38:57 rsi-test-c7 systemd: Started LSB: cloud-dhcp-check.
Apr 5 13:38:57 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-sshkey...
Apr 5 13:38:57 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-password...
Apr 5 13:38:57 rsi-test-c7 cloud: Found ssh key server at 10.1.1.1
Apr 5 13:38:57 rsi-test-c7 cloud: Found password server IP 10.1.1.1 in /var/lib/dhclient/dhclient.leases
Apr 5 13:38:57 rsi-test-c7 cloud: Sending request to ssh key server at 10.1.1.1
Apr 5 13:38:57 rsi-test-c7 cloud: Sending request to password server at 10.1.1.1
Apr 5 13:38:57 rsi-test-c7 cloud: Got response from server at 10.1.1.1
Apr 5 13:38:57 rsi-test-c7 cloud: Did not receive any keys from any server
Apr 5 13:38:57 rsi-test-c7 systemd: cloud-set-guest-sshkey.service: control process exited, code=exited status=1
Apr 5 13:38:57 rsi-test-c7 systemd: Failed to start LSB: cloud-set-guest-sshkey.
Apr 5 13:38:57 rsi-test-c7 systemd: Unit cloud-set-guest-sshkey.service entered failed state.
Apr 5 13:38:57 rsi-test-c7 systemd: cloud-set-guest-sshkey.service failed.
Apr 5 13:38:57 rsi-test-c7 cloud: Got response from server at 10.1.1.1
Apr 5 13:38:57 rsi-test-c7 cloud: VM got a valid password from server at 10.1.1.1
Apr 5 13:38:57 rsi-test-c7 cloud: Changing password for user root
Apr 5 13:38:57 rsi-test-c7 cloud: Sending acknowledgment to password server at 10.1.1.1
Apr 5 13:38:57 rsi-test-c7 cloud-set-guest-password: saved_password
Apr 5 13:38:57 rsi-test-c7 systemd: Started LSB: cloud-set-guest-password.
Apr 5 13:38:57 rsi-test-c7 systemd: Starting Initial cloud-init job (metadata service crawler)...
Apr 5 13:38:58 rsi-test-c7 systemd: Started Initial cloud-init job (metadata service crawler).
Apr 5 13:38:58 rsi-test-c7 systemd: Starting Apply the settings specified in cloud-config...
Apr 5 13:38:58 rsi-test-c7 systemd: Started Apply the settings specified in cloud-config.
Apr 5 13:38:58 rsi-test-c7 systemd: Starting Execute cloud user/final scripts...
Apr 5 13:38:59 rsi-test-c7 systemd: Started Execute cloud user/final scripts.
Let me know if I've missed something or you need further info.
Richard Klein <rk...@rsitex.com>
RSI
5426 Guadalupe, Suite 100
Austin TX 78751
> -----Original Message-----
> From: Erik Weber [mailto:terbolous@gmail.com]
> Sent: Tuesday, April 05, 2016 12:30 AM
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack 4.7 password reset issue.
>
> On Mon, Apr 4, 2016 at 9:17 PM, Richard Klein (RSI) <rk...@rsitex.com>
> wrote:
>
> > Thanks for the reply and sorry for the delay. You know how it is on
> > Monday's.
> >
> > I checked the VR password file as suggested after the password reset
> > via the GUI but before powering on the VM. I did see the password
> > reset there as expected. I powered on the VM and the password was reset as
> expected.
> > The test VM is built from a CentOS7 template with password enabled. I
> > checked the /var/log/messages and you can see from the client
> > perspective the change went as expected. Below is "snippet1" from the
> > message log on the VM (there are no SSH key so it fails as expected):
> >
> >
> This is your first VM snippet:
>
>
> > Apr 4 13:45:35 rsi-test-c7 systemd: Starting LSB:
> > cloud-set-guest-sshkey...
> > Apr 4 13:45:35 rsi-test-c7 systemd: Starting LSB:
> > cloud-set-guest-password...
> > Apr 4 13:45:35 rsi-test-c7 cloud: Unable to determine the password
> > server, falling back to data-server Apr 4 13:45:35 rsi-test-c7 cloud:
> > Could not find ssh key server IP in /var/lib/dhclient/dhclient.leases
> > Apr 4 13:45:36 rsi-test-c7 cloud: Unable to determine the password
> > server, falling back to data-server Apr 4 13:45:36 rsi-test-c7 cloud:
> > Sending request to password server at data-server Apr 4 13:45:36
> > rsi-test-c7 cloud: Sending request to ssh key server at data-server
> > Apr 4 13:45:36 rsi-test-c7 cloud: Got response from server at
> > data-server Apr 4 13:45:36 rsi-test-c7 cloud: Did not receive any
> > keys from any server Apr 4 13:45:36 rsi-test-c7 systemd: cloud-set-guest-
> sshkey.service:
> > control process exited, code=exited status=1 Apr 4 13:45:36
> > rsi-test-c7 systemd: Failed to start LSB:
> > cloud-set-guest-sshkey.
> > Apr 4 13:45:36 rsi-test-c7 systemd: Unit
> > cloud-set-guest-sshkey.service entered failed state.
> > Apr 4 13:45:36 rsi-test-c7 systemd: cloud-set-guest-sshkey.service failed.
> > Apr 4 13:45:36 rsi-test-c7 cloud: Got response from server at
> > data-server Apr 4 13:45:36 rsi-test-c7 cloud: VM got a valid password
> > from server at data-server Apr 4 13:45:36 rsi-test-c7 cloud: Changing
> > password for user root Apr 4 13:45:36 rsi-test-c7 cloud: Sending
> > acknowledgment to password server at data-server Apr 4 13:45:36
> > rsi-test-c7 cloud-set-guest-password: saved_password Apr 4 13:45:36
> > rsi-test-c7 systemd: Started LSB: cloud-set-guest-password.
> > Apr 4 13:45:36 rsi-test-c7 systemd: Starting Initial cloud-init job
> > (metadata service crawler)...
> > Apr 4 13:45:37 rsi-test-c7 systemd: Started Initial cloud-init job
> > (metadata service crawler).
> > Apr 4 13:45:37 rsi-test-c7 systemd: Starting Apply the settings
> > specified in cloud-config...
> > Apr 4 13:45:37 rsi-test-c7 systemd: Started Apply the settings
> > specified in cloud-config.
> > Apr 4 13:45:37 rsi-test-c7 systemd: Starting Execute cloud user/final
> > scripts...
> > Apr 4 13:45:37 rsi-test-c7 systemd: Started Execute cloud user/final
> > scripts.
> >
> > I checked the password file on the VR and it was empty now. I then
> > manually reset the password on the VM console only (not CS GUI) and
> > performed an OS reboot from the console. I could log in with the
> > password I change with the console. Below is the system messages
> "snippet2":
> >
> >
> This is your second VM snippet, happening before snippet1
>
>
> > Apr 4 13:36:27 rsi-test-c7 systemd: Starting LSB:
> > cloud-set-guest-sshkey...
> > Apr 4 13:36:27 rsi-test-c7 systemd: Starting LSB:
> > cloud-set-guest-password...
> > Apr 4 13:36:27 rsi-test-c7 cloud: Could not find ssh key server IP in
> > /var/lib/dhclient/dhclient.leases
> >
>
> Do you have such file, or is there a configuration error in the cloud-set-guest-
> password?
> You seem to be running centos7, and that wasn't around when these scripts
> where made and thus might have to be modified for them to work.
>
>
> > Apr 4 13:36:27 rsi-test-c7 cloud: Unable to determine the password
> > server, falling back to data-server Apr 4 13:36:27 rsi-test-c7 cloud:
> > Unable to determine the password server, falling back to data-server
> > Apr 4 13:36:27 rsi-test-c7 cloud: Sending request to password server
> > at data-server Apr 4 13:36:27 rsi-test-c7 cloud: Sending request to
> > ssh key server at data-server Apr 4 13:36:27 rsi-test-c7 cloud: Got
> > response from server at data-server Apr 4 13:36:27 rsi-test-c7 cloud:
> > Did not receive any keys from any server Apr 4 13:36:27 rsi-test-c7
> > systemd: cloud-set-guest-sshkey.service:
> > control process exited, code=exited status=1 Apr 4 13:36:27
> > rsi-test-c7 systemd: Failed to start LSB:
> > cloud-set-guest-sshkey.
> > Apr 4 13:36:27 rsi-test-c7 systemd: Unit
> > cloud-set-guest-sshkey.service entered failed state.
> > Apr 4 13:36:27 rsi-test-c7 systemd: cloud-set-guest-sshkey.service failed.
> > Apr 4 13:36:27 rsi-test-c7 cloud: Got response from server at
> > data-server Apr 4 13:36:27 rsi-test-c7 cloud: VM has already saved a
> > password from the password server at data-server Apr 4 13:36:27
> > rsi-test-c7 cloud: Did not need to change password.
> > Apr 4 13:36:27 rsi-test-c7 systemd: Started LSB: cloud-set-guest-password.
> >
> > I then powered off the VM and powered it back on without changing any
> > passwords. The password was set to the last change password via the GUI.
> > The logs behave as if the password reset was performed from the GUI
> > again (snippet1). I checked the password file on the VR and it was
> > empty. I also monitored the VR /var/log/cloud.log messages and
> > discovered something interesting. As expected there was a flurry of
> > activity but the following
> > 2 messages stood out to me:
> >
> >
> This is your VR snippet.
>
>
> > 2016-04-04 18:45:06,937 merge.py save:72 {u'id': u'dhcpentry',
> > u'10.1.1.39': {u'default_gateway': u'10.1.1.1', u'ipv6_duid': u'00:03:00:01:
> > 02:00:26:c0:00:07', u'default_entry': True, u'ipv4_adress':
> > u'10.1.1.39',
> > u'host_name': u'rsi-test-c7', u'mac_address': u'02:00:26:c0:00:07'
> > , u'type': u'dhcpentry', u'dns_adresses': u'10.1.1.1'}}
> >
> > 2016-04-04 18:45:07,006 CsHelper.py execute:160 Executing: curl
> > --header
> > "DomU_Request: save_password" "http://10.1.1.1:8080/" -F "ip=10.1.1.39"
> > -F "password=a8Q7Bs" -F "token=ae199529efcd26f7811f36ae7c0807b1"
> > >/dev/null
> > 2>/dev/null &
> >
> >
> I am assuming you have a timezone difference between the VR and your VMs,
> but could you confirm that the VR time of 18:45 is 13:45 your time?
>
> But if so, the VR snippet you pasted is happening BEFORE the first VM snippet
> and looks correct.
>
> It would be helpful if you also could attach the logs for the duration of the first
> VM snippet.
> Also, could you point us to the source of the cloud-set-guest-* scripts you used,
> preferably with any modifications?
>
>
> --
> Erik
Re: Cloudstack 4.7 password reset issue.
Posted by Erik Weber <te...@gmail.com>.
On Mon, Apr 4, 2016 at 9:17 PM, Richard Klein (RSI) <rk...@rsitex.com>
wrote:
> Thanks for the reply and sorry for the delay. You know how it is on
> Monday's.
>
> I checked the VR password file as suggested after the password reset via
> the GUI but before powering on the VM. I did see the password reset there
> as expected. I powered on the VM and the password was reset as expected.
> The test VM is built from a CentOS7 template with password enabled. I
> checked the /var/log/messages and you can see from the client perspective
> the change went as expected. Below is "snippet1" from the message log on
> the VM (there are no SSH key so it fails as expected):
>
>
This is your first VM snippet:
> Apr 4 13:45:35 rsi-test-c7 systemd: Starting LSB:
> cloud-set-guest-sshkey...
> Apr 4 13:45:35 rsi-test-c7 systemd: Starting LSB:
> cloud-set-guest-password...
> Apr 4 13:45:35 rsi-test-c7 cloud: Unable to determine the password
> server, falling back to data-server
> Apr 4 13:45:35 rsi-test-c7 cloud: Could not find ssh key server IP in
> /var/lib/dhclient/dhclient.leases
> Apr 4 13:45:36 rsi-test-c7 cloud: Unable to determine the password
> server, falling back to data-server
> Apr 4 13:45:36 rsi-test-c7 cloud: Sending request to password server at
> data-server
> Apr 4 13:45:36 rsi-test-c7 cloud: Sending request to ssh key server at
> data-server
> Apr 4 13:45:36 rsi-test-c7 cloud: Got response from server at data-server
> Apr 4 13:45:36 rsi-test-c7 cloud: Did not receive any keys from any server
> Apr 4 13:45:36 rsi-test-c7 systemd: cloud-set-guest-sshkey.service:
> control process exited, code=exited status=1
> Apr 4 13:45:36 rsi-test-c7 systemd: Failed to start LSB:
> cloud-set-guest-sshkey.
> Apr 4 13:45:36 rsi-test-c7 systemd: Unit cloud-set-guest-sshkey.service
> entered failed state.
> Apr 4 13:45:36 rsi-test-c7 systemd: cloud-set-guest-sshkey.service failed.
> Apr 4 13:45:36 rsi-test-c7 cloud: Got response from server at data-server
> Apr 4 13:45:36 rsi-test-c7 cloud: VM got a valid password from server at
> data-server
> Apr 4 13:45:36 rsi-test-c7 cloud: Changing password for user root
> Apr 4 13:45:36 rsi-test-c7 cloud: Sending acknowledgment to password
> server at data-server
> Apr 4 13:45:36 rsi-test-c7 cloud-set-guest-password: saved_password
> Apr 4 13:45:36 rsi-test-c7 systemd: Started LSB: cloud-set-guest-password.
> Apr 4 13:45:36 rsi-test-c7 systemd: Starting Initial cloud-init job
> (metadata service crawler)...
> Apr 4 13:45:37 rsi-test-c7 systemd: Started Initial cloud-init job
> (metadata service crawler).
> Apr 4 13:45:37 rsi-test-c7 systemd: Starting Apply the settings specified
> in cloud-config...
> Apr 4 13:45:37 rsi-test-c7 systemd: Started Apply the settings specified
> in cloud-config.
> Apr 4 13:45:37 rsi-test-c7 systemd: Starting Execute cloud user/final
> scripts...
> Apr 4 13:45:37 rsi-test-c7 systemd: Started Execute cloud user/final
> scripts.
>
> I checked the password file on the VR and it was empty now. I then
> manually reset the password on the VM console only (not CS GUI) and
> performed an OS reboot from the console. I could log in with the password
> I change with the console. Below is the system messages "snippet2":
>
>
This is your second VM snippet, happening before snippet1
> Apr 4 13:36:27 rsi-test-c7 systemd: Starting LSB:
> cloud-set-guest-sshkey...
> Apr 4 13:36:27 rsi-test-c7 systemd: Starting LSB:
> cloud-set-guest-password...
> Apr 4 13:36:27 rsi-test-c7 cloud: Could not find ssh key server IP in
> /var/lib/dhclient/dhclient.leases
>
Do you have such file, or is there a configuration error in the
cloud-set-guest-password?
You seem to be running centos7, and that wasn't around when these scripts
where made and thus might have to be modified for them to work.
> Apr 4 13:36:27 rsi-test-c7 cloud: Unable to determine the password
> server, falling back to data-server
> Apr 4 13:36:27 rsi-test-c7 cloud: Unable to determine the password
> server, falling back to data-server
> Apr 4 13:36:27 rsi-test-c7 cloud: Sending request to password server at
> data-server
> Apr 4 13:36:27 rsi-test-c7 cloud: Sending request to ssh key server at
> data-server
> Apr 4 13:36:27 rsi-test-c7 cloud: Got response from server at data-server
> Apr 4 13:36:27 rsi-test-c7 cloud: Did not receive any keys from any server
> Apr 4 13:36:27 rsi-test-c7 systemd: cloud-set-guest-sshkey.service:
> control process exited, code=exited status=1
> Apr 4 13:36:27 rsi-test-c7 systemd: Failed to start LSB:
> cloud-set-guest-sshkey.
> Apr 4 13:36:27 rsi-test-c7 systemd: Unit cloud-set-guest-sshkey.service
> entered failed state.
> Apr 4 13:36:27 rsi-test-c7 systemd: cloud-set-guest-sshkey.service failed.
> Apr 4 13:36:27 rsi-test-c7 cloud: Got response from server at data-server
> Apr 4 13:36:27 rsi-test-c7 cloud: VM has already saved a password from
> the password server at data-server
> Apr 4 13:36:27 rsi-test-c7 cloud: Did not need to change password.
> Apr 4 13:36:27 rsi-test-c7 systemd: Started LSB: cloud-set-guest-password.
>
> I then powered off the VM and powered it back on without changing any
> passwords. The password was set to the last change password via the GUI.
> The logs behave as if the password reset was performed from the GUI again
> (snippet1). I checked the password file on the VR and it was empty. I
> also monitored the VR /var/log/cloud.log messages and discovered something
> interesting. As expected there was a flurry of activity but the following
> 2 messages stood out to me:
>
>
This is your VR snippet.
> 2016-04-04 18:45:06,937 merge.py save:72 {u'id': u'dhcpentry',
> u'10.1.1.39': {u'default_gateway': u'10.1.1.1', u'ipv6_duid': u'00:03:00:01:
> 02:00:26:c0:00:07', u'default_entry': True, u'ipv4_adress': u'10.1.1.39',
> u'host_name': u'rsi-test-c7', u'mac_address': u'02:00:26:c0:00:07'
> , u'type': u'dhcpentry', u'dns_adresses': u'10.1.1.1'}}
>
> 2016-04-04 18:45:07,006 CsHelper.py execute:160 Executing: curl --header
> "DomU_Request: save_password" "http://10.1.1.1:8080/" -F "ip=10.1.1.39"
> -F "password=a8Q7Bs" -F "token=ae199529efcd26f7811f36ae7c0807b1" >/dev/null
> 2>/dev/null &
>
>
I am assuming you have a timezone difference between the VR and your VMs,
but could you confirm that the VR time of 18:45 is 13:45 your time?
But if so, the VR snippet you pasted is happening BEFORE the first VM
snippet and looks correct.
It would be helpful if you also could attach the logs for the duration of
the first VM snippet.
Also, could you point us to the source of the cloud-set-guest-* scripts you
used, preferably with any modifications?
--
Erik
Re: Cloudstack 4.7 password reset issue.
Posted by Shweta Agarwal <sh...@accelerite.com>.
Hi Richard,
This looks like a bug to me. The password should be pushed only first time of VM start and only after User reset password of VM via CS.
Thanks
Shweta
Principal Product Engineer, CloudPlatform
Accelerite
________________________________________
From: Richard Klein (RSI) <rk...@rsitex.com>
Sent: Tuesday, April 5, 2016 12:47 AM
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.7 password reset issue.
Thanks for the reply and sorry for the delay. You know how it is on Monday's.
I checked the VR password file as suggested after the password reset via the GUI but before powering on the VM. I did see the password reset there as expected. I powered on the VM and the password was reset as expected. The test VM is built from a CentOS7 template with password enabled. I checked the /var/log/messages and you can see from the client perspective the change went as expected. Below is "snippet1" from the message log on the VM (there are no SSH key so it fails as expected):
Apr 4 13:45:35 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-sshkey...
Apr 4 13:45:35 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-password...
Apr 4 13:45:35 rsi-test-c7 cloud: Unable to determine the password server, falling back to data-server
Apr 4 13:45:35 rsi-test-c7 cloud: Could not find ssh key server IP in /var/lib/dhclient/dhclient.leases
Apr 4 13:45:36 rsi-test-c7 cloud: Unable to determine the password server, falling back to data-server
Apr 4 13:45:36 rsi-test-c7 cloud: Sending request to password server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud: Sending request to ssh key server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud: Got response from server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud: Did not receive any keys from any server
Apr 4 13:45:36 rsi-test-c7 systemd: cloud-set-guest-sshkey.service: control process exited, code=exited status=1
Apr 4 13:45:36 rsi-test-c7 systemd: Failed to start LSB: cloud-set-guest-sshkey.
Apr 4 13:45:36 rsi-test-c7 systemd: Unit cloud-set-guest-sshkey.service entered failed state.
Apr 4 13:45:36 rsi-test-c7 systemd: cloud-set-guest-sshkey.service failed.
Apr 4 13:45:36 rsi-test-c7 cloud: Got response from server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud: VM got a valid password from server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud: Changing password for user root
Apr 4 13:45:36 rsi-test-c7 cloud: Sending acknowledgment to password server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud-set-guest-password: saved_password
Apr 4 13:45:36 rsi-test-c7 systemd: Started LSB: cloud-set-guest-password.
Apr 4 13:45:36 rsi-test-c7 systemd: Starting Initial cloud-init job (metadata service crawler)...
Apr 4 13:45:37 rsi-test-c7 systemd: Started Initial cloud-init job (metadata service crawler).
Apr 4 13:45:37 rsi-test-c7 systemd: Starting Apply the settings specified in cloud-config...
Apr 4 13:45:37 rsi-test-c7 systemd: Started Apply the settings specified in cloud-config.
Apr 4 13:45:37 rsi-test-c7 systemd: Starting Execute cloud user/final scripts...
Apr 4 13:45:37 rsi-test-c7 systemd: Started Execute cloud user/final scripts.
I checked the password file on the VR and it was empty now. I then manually reset the password on the VM console only (not CS GUI) and performed an OS reboot from the console. I could log in with the password I change with the console. Below is the system messages "snippet2":
Apr 4 13:36:27 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-sshkey...
Apr 4 13:36:27 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-password...
Apr 4 13:36:27 rsi-test-c7 cloud: Could not find ssh key server IP in /var/lib/dhclient/dhclient.leases
Apr 4 13:36:27 rsi-test-c7 cloud: Unable to determine the password server, falling back to data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Unable to determine the password server, falling back to data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Sending request to password server at data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Sending request to ssh key server at data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Got response from server at data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Did not receive any keys from any server
Apr 4 13:36:27 rsi-test-c7 systemd: cloud-set-guest-sshkey.service: control process exited, code=exited status=1
Apr 4 13:36:27 rsi-test-c7 systemd: Failed to start LSB: cloud-set-guest-sshkey.
Apr 4 13:36:27 rsi-test-c7 systemd: Unit cloud-set-guest-sshkey.service entered failed state.
Apr 4 13:36:27 rsi-test-c7 systemd: cloud-set-guest-sshkey.service failed.
Apr 4 13:36:27 rsi-test-c7 cloud: Got response from server at data-server
Apr 4 13:36:27 rsi-test-c7 cloud: VM has already saved a password from the password server at data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Did not need to change password.
Apr 4 13:36:27 rsi-test-c7 systemd: Started LSB: cloud-set-guest-password.
I then powered off the VM and powered it back on without changing any passwords. The password was set to the last change password via the GUI. The logs behave as if the password reset was performed from the GUI again (snippet1). I checked the password file on the VR and it was empty. I also monitored the VR /var/log/cloud.log messages and discovered something interesting. As expected there was a flurry of activity but the following 2 messages stood out to me:
2016-04-04 18:45:06,937 merge.py save:72 {u'id': u'dhcpentry', u'10.1.1.39': {u'default_gateway': u'10.1.1.1', u'ipv6_duid': u'00:03:00:01:
02:00:26:c0:00:07', u'default_entry': True, u'ipv4_adress': u'10.1.1.39', u'host_name': u'rsi-test-c7', u'mac_address': u'02:00:26:c0:00:07'
, u'type': u'dhcpentry', u'dns_adresses': u'10.1.1.1'}}
2016-04-04 18:45:07,006 CsHelper.py execute:160 Executing: curl --header "DomU_Request: save_password" "http://10.1.1.1:8080/" -F "ip=10.1.1.39" -F "password=a8Q7Bs" -F "token=ae199529efcd26f7811f36ae7c0807b1" >/dev/null 2>/dev/null &
The VM I powered on IP address is 10.1.1.39 and it appears to me that the CsHelper.py curl command is setting the password again? Making an educated guess it looks like the management process is sending the reset password again to the VR after the power up?
Please let me know what you think or if you need further information just let me know. Thanks.
----
Richard Klein <rk...@rsitex.com>
RSI
5426 Guadalupe, Suite 100
Austin TX 78751
>-----Original Message-----
>From: Suresh Sadhu [mailto:suresh.sadhu@accelerite.com]
>Sent: Monday, April 04, 2016 5:21 AM
>To: users@cloudstack.apache.org
>Subject: RE: Cloudstack 4.7 password reset issue.
>
>This look like an issue, please check the VR password file (/var/cache/cloud/passwords) and entries look like below: Guest_vm_ip=Password)
>Eg:
>10.1.1.145=sX3qvbstr --- here, In this case it successfully generated the password but didn't applied on Guest vm
>10.1.1.135=saved_password -- if it applied successfully on VM then it will show as guest_vm_ip=Saved_password.)
>
>So please check you entries as well check your logs for any errors.
>
>Regards
>Sadhu
>Chief Product Engineer, Accelerite
>Suresh.sadhu@accelerite.com
>>-----Original Message-----
>>From: Erik Weber [mailto:terbolous@gmail.com]
>>Sent: Monday, April 4, 2016 11:54 AM
>>To: users@cloudstack.apache.org
>>Subject: Re: Cloudstack 4.7 password reset issue.
>>
>>On Mon, Apr 4, 2016 at 7:28 AM, Shweta Agarwal < shweta.agarwal1@accelerite.com> wrote:
>>
>> Hi Richard,
>> Cloudstack is working as expected at every steps you mentioned .
>>
>> when your template is password enabled then every time you reset
>> password via CS and stop start the VM then router will push new
>> password to the user VM .
>>
This works, but if you read his example you will see that the password is reset after an additional reboot, which it should not.
Looks like a bug, but more info is needed to state if it is in CloudStack or the password reset script.
--
Erik
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
RE: Cloudstack 4.7 password reset issue.
Posted by "Richard Klein (RSI)" <rk...@rsitex.com>.
Thanks for the reply and sorry for the delay. You know how it is on Monday's.
I checked the VR password file as suggested after the password reset via the GUI but before powering on the VM. I did see the password reset there as expected. I powered on the VM and the password was reset as expected. The test VM is built from a CentOS7 template with password enabled. I checked the /var/log/messages and you can see from the client perspective the change went as expected. Below is "snippet1" from the message log on the VM (there are no SSH key so it fails as expected):
Apr 4 13:45:35 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-sshkey...
Apr 4 13:45:35 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-password...
Apr 4 13:45:35 rsi-test-c7 cloud: Unable to determine the password server, falling back to data-server
Apr 4 13:45:35 rsi-test-c7 cloud: Could not find ssh key server IP in /var/lib/dhclient/dhclient.leases
Apr 4 13:45:36 rsi-test-c7 cloud: Unable to determine the password server, falling back to data-server
Apr 4 13:45:36 rsi-test-c7 cloud: Sending request to password server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud: Sending request to ssh key server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud: Got response from server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud: Did not receive any keys from any server
Apr 4 13:45:36 rsi-test-c7 systemd: cloud-set-guest-sshkey.service: control process exited, code=exited status=1
Apr 4 13:45:36 rsi-test-c7 systemd: Failed to start LSB: cloud-set-guest-sshkey.
Apr 4 13:45:36 rsi-test-c7 systemd: Unit cloud-set-guest-sshkey.service entered failed state.
Apr 4 13:45:36 rsi-test-c7 systemd: cloud-set-guest-sshkey.service failed.
Apr 4 13:45:36 rsi-test-c7 cloud: Got response from server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud: VM got a valid password from server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud: Changing password for user root
Apr 4 13:45:36 rsi-test-c7 cloud: Sending acknowledgment to password server at data-server
Apr 4 13:45:36 rsi-test-c7 cloud-set-guest-password: saved_password
Apr 4 13:45:36 rsi-test-c7 systemd: Started LSB: cloud-set-guest-password.
Apr 4 13:45:36 rsi-test-c7 systemd: Starting Initial cloud-init job (metadata service crawler)...
Apr 4 13:45:37 rsi-test-c7 systemd: Started Initial cloud-init job (metadata service crawler).
Apr 4 13:45:37 rsi-test-c7 systemd: Starting Apply the settings specified in cloud-config...
Apr 4 13:45:37 rsi-test-c7 systemd: Started Apply the settings specified in cloud-config.
Apr 4 13:45:37 rsi-test-c7 systemd: Starting Execute cloud user/final scripts...
Apr 4 13:45:37 rsi-test-c7 systemd: Started Execute cloud user/final scripts.
I checked the password file on the VR and it was empty now. I then manually reset the password on the VM console only (not CS GUI) and performed an OS reboot from the console. I could log in with the password I change with the console. Below is the system messages "snippet2":
Apr 4 13:36:27 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-sshkey...
Apr 4 13:36:27 rsi-test-c7 systemd: Starting LSB: cloud-set-guest-password...
Apr 4 13:36:27 rsi-test-c7 cloud: Could not find ssh key server IP in /var/lib/dhclient/dhclient.leases
Apr 4 13:36:27 rsi-test-c7 cloud: Unable to determine the password server, falling back to data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Unable to determine the password server, falling back to data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Sending request to password server at data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Sending request to ssh key server at data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Got response from server at data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Did not receive any keys from any server
Apr 4 13:36:27 rsi-test-c7 systemd: cloud-set-guest-sshkey.service: control process exited, code=exited status=1
Apr 4 13:36:27 rsi-test-c7 systemd: Failed to start LSB: cloud-set-guest-sshkey.
Apr 4 13:36:27 rsi-test-c7 systemd: Unit cloud-set-guest-sshkey.service entered failed state.
Apr 4 13:36:27 rsi-test-c7 systemd: cloud-set-guest-sshkey.service failed.
Apr 4 13:36:27 rsi-test-c7 cloud: Got response from server at data-server
Apr 4 13:36:27 rsi-test-c7 cloud: VM has already saved a password from the password server at data-server
Apr 4 13:36:27 rsi-test-c7 cloud: Did not need to change password.
Apr 4 13:36:27 rsi-test-c7 systemd: Started LSB: cloud-set-guest-password.
I then powered off the VM and powered it back on without changing any passwords. The password was set to the last change password via the GUI. The logs behave as if the password reset was performed from the GUI again (snippet1). I checked the password file on the VR and it was empty. I also monitored the VR /var/log/cloud.log messages and discovered something interesting. As expected there was a flurry of activity but the following 2 messages stood out to me:
2016-04-04 18:45:06,937 merge.py save:72 {u'id': u'dhcpentry', u'10.1.1.39': {u'default_gateway': u'10.1.1.1', u'ipv6_duid': u'00:03:00:01:
02:00:26:c0:00:07', u'default_entry': True, u'ipv4_adress': u'10.1.1.39', u'host_name': u'rsi-test-c7', u'mac_address': u'02:00:26:c0:00:07'
, u'type': u'dhcpentry', u'dns_adresses': u'10.1.1.1'}}
2016-04-04 18:45:07,006 CsHelper.py execute:160 Executing: curl --header "DomU_Request: save_password" "http://10.1.1.1:8080/" -F "ip=10.1.1.39" -F "password=a8Q7Bs" -F "token=ae199529efcd26f7811f36ae7c0807b1" >/dev/null 2>/dev/null &
The VM I powered on IP address is 10.1.1.39 and it appears to me that the CsHelper.py curl command is setting the password again? Making an educated guess it looks like the management process is sending the reset password again to the VR after the power up?
Please let me know what you think or if you need further information just let me know. Thanks.
----
Richard Klein <rk...@rsitex.com>
RSI
5426 Guadalupe, Suite 100
Austin TX 78751
>-----Original Message-----
>From: Suresh Sadhu [mailto:suresh.sadhu@accelerite.com]
>Sent: Monday, April 04, 2016 5:21 AM
>To: users@cloudstack.apache.org
>Subject: RE: Cloudstack 4.7 password reset issue.
>
>This look like an issue, please check the VR password file (/var/cache/cloud/passwords) and entries look like below: Guest_vm_ip=Password)
>Eg:
>10.1.1.145=sX3qvbstr --- here, In this case it successfully generated the password but didn't applied on Guest vm
>10.1.1.135=saved_password -- if it applied successfully on VM then it will show as guest_vm_ip=Saved_password.)
>
>So please check you entries as well check your logs for any errors.
>
>Regards
>Sadhu
>Chief Product Engineer, Accelerite
>Suresh.sadhu@accelerite.com
>>-----Original Message-----
>>From: Erik Weber [mailto:terbolous@gmail.com]
>>Sent: Monday, April 4, 2016 11:54 AM
>>To: users@cloudstack.apache.org
>>Subject: Re: Cloudstack 4.7 password reset issue.
>>
>>On Mon, Apr 4, 2016 at 7:28 AM, Shweta Agarwal < shweta.agarwal1@accelerite.com> wrote:
>>
>> Hi Richard,
>> Cloudstack is working as expected at every steps you mentioned .
>>
>> when your template is password enabled then every time you reset
>> password via CS and stop start the VM then router will push new
>> password to the user VM .
>>
This works, but if you read his example you will see that the password is reset after an additional reboot, which it should not.
Looks like a bug, but more info is needed to state if it is in CloudStack or the password reset script.
--
Erik
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
RE: Cloudstack 4.7 password reset issue.
Posted by Suresh Sadhu <su...@accelerite.com>.
This look like an issue, please check the VR password file (/var/cache/cloud/passwords) and entries look like below: Guest_vm_ip=Password)
Eg:
10.1.1.145=sX3qvbstr --- here, In this case it successfully generated the password but didn't applied on Guest vm
10.1.1.135=saved_password -- if it applied successfully on VM then it will show as guest_vm_ip=Saved_password.)
So please check you entries as well check your logs for any errors.
Regards
Sadhu
Chief Product Engineer, Accelerite
Suresh.sadhu@accelerite.com
-----Original Message-----
From: Erik Weber [mailto:terbolous@gmail.com]
Sent: Monday, April 4, 2016 11:54 AM
To: users@cloudstack.apache.org
Subject: Re: Cloudstack 4.7 password reset issue.
On Mon, Apr 4, 2016 at 7:28 AM, Shweta Agarwal < shweta.agarwal1@accelerite.com> wrote:
> Hi Richard,
> Cloudstack is working as expected at every steps you mentioned .
>
> when your template is password enabled then every time you reset
> password via CS and stop start the VM then router will push new
> password to the user VM .
>
This works, but if you read his example you will see that the password is reset after an additional reboot, which it should not.
Looks like a bug, but more info is needed to state if it is in CloudStack or the password reset script.
--
Erik
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: Cloudstack 4.7 password reset issue.
Posted by Erik Weber <te...@gmail.com>.
On Mon, Apr 4, 2016 at 7:28 AM, Shweta Agarwal <
shweta.agarwal1@accelerite.com> wrote:
> Hi Richard,
> Cloudstack is working as expected at every steps you mentioned .
>
> when your template is password enabled then every time you reset password
> via CS and stop start the VM then router will push new password to the user
> VM .
>
This works, but if you read his example you will see that the password is
reset after an additional reboot, which it should not.
Looks like a bug, but more info is needed to state if it is in CloudStack
or the password reset script.
--
Erik
Re: Cloudstack 4.7 password reset issue.
Posted by Shweta Agarwal <sh...@accelerite.com>.
Hi Richard,
Cloudstack is working as expected at every steps you mentioned .
when your template is password enabled then every time you reset password via CS and stop start the VM then router will push new password to the user VM .
If you want a VM with your set password then create a VM with a template having no password enabled check . In that case when you login to VM and change then password via VM console then on stop start of VM the password will remain what you set via consol.
Hope it helps to understand thing
Thanks
Shweta
Principal Product Engineer, CloudPlatform
Accelerite
________________________________________
From: Erik Weber <te...@gmail.com>
Sent: Sunday, April 3, 2016 12:40 AM
To: users@cloudstack.apache.org
Subject: Re: Cloudstack 4.7 password reset issue.
On Sat, Apr 2, 2016 at 9:04 PM, Richard Klein (RSI) <rk...@rsitex.com>
wrote:
> First time mailer here, hello everyone.
Hi, and welcome!
> We are implementing CS 4.7 cluster with CentOS7. All is going well with a
> few bumps that we have resolved. During testing we are getting an unusual
> behavior from the password reset process. We have setup the templates
> using the instructions from the documentation including installing the
> guest password and SSH setup scripts. Below is what happens:
>
> * Create a VM from template that has the password option checked and the
> root password is set as expected.
> * Log in with 1st password and then change password of root via the
> console of the VM
> * Perform a reboot from the VM console and the password stays as set
> manually.
> * Power off VM, change PW with CS web GUI. When the VM is powered on the
> password is the new one (as expected).
> * Log in with new password on the console and then manually change root
> again.
> * Reboot from the console and the password remains as manually set.
> * Power off VM via web GUI and then power it back on.
> * The password reverts to the last password reset operation from the web
> GUI.
>
> I've looked at the message logs and the guest password and SSH scripts are
> running normally. The virtual router is sending the password reset again
> when the service query it. But this only occurs after the VM is powered
> off then on again. I expected the password to remain as the manually set
> password but it always reverts to the last password reset operation when
> the VM is powered off then on again.
>
Check the logs on the relevant VR. If the password script does not
successfully send the acknowledgment that the password was saved it won't
get removed from the VR and be set on subsequent reboots.
Don't recall at the moment if is /var/log/cloud.log, /var/log/syslog, but
just look around.
--
Erik
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: Cloudstack 4.7 password reset issue.
Posted by Erik Weber <te...@gmail.com>.
On Sat, Apr 2, 2016 at 9:04 PM, Richard Klein (RSI) <rk...@rsitex.com>
wrote:
> First time mailer here, hello everyone.
Hi, and welcome!
> We are implementing CS 4.7 cluster with CentOS7. All is going well with a
> few bumps that we have resolved. During testing we are getting an unusual
> behavior from the password reset process. We have setup the templates
> using the instructions from the documentation including installing the
> guest password and SSH setup scripts. Below is what happens:
>
> * Create a VM from template that has the password option checked and the
> root password is set as expected.
> * Log in with 1st password and then change password of root via the
> console of the VM
> * Perform a reboot from the VM console and the password stays as set
> manually.
> * Power off VM, change PW with CS web GUI. When the VM is powered on the
> password is the new one (as expected).
> * Log in with new password on the console and then manually change root
> again.
> * Reboot from the console and the password remains as manually set.
> * Power off VM via web GUI and then power it back on.
> * The password reverts to the last password reset operation from the web
> GUI.
>
> I've looked at the message logs and the guest password and SSH scripts are
> running normally. The virtual router is sending the password reset again
> when the service query it. But this only occurs after the VM is powered
> off then on again. I expected the password to remain as the manually set
> password but it always reverts to the last password reset operation when
> the VM is powered off then on again.
>
Check the logs on the relevant VR. If the password script does not
successfully send the acknowledgment that the password was saved it won't
get removed from the VR and be set on subsequent reboots.
Don't recall at the moment if is /var/log/cloud.log, /var/log/syslog, but
just look around.
--
Erik