You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by "Richard Klein (RSI)" <rk...@rsitex.com> on 2016/04/13 22:22:50 UTC
RE: Cloudstack 4.7 password reset issue - resolved.
I finally found the problem and resolved the issue. The problem was in the Python code change I made. I had a flag variable that indicated to save data when it was changed while processing a list. This worked fine as long as it executed the logic and defined the flag variable. The problem was during startup when it doesn't go through the loop and the flag variable was undefined. This cause the "update_config.py" to fail which then bubbled back up as an error and prevent the router from starting.
Once I changed the code and rebuilt the project all worked well and the bug is fixed. Thanks so much for everyone's help. This process was very educational and looking forward to learning more.
I do have one question just out of curiosity. What makes the "Requires Upgrade" column on the Home->Infrastructure->Virtual Router page indicated "Yes"?
Thanks again,
Richard Klein <rk...@rsitex.com>
RSI
5426 Guadalupe, Suite 100
Austin TX 78751
> -----Original Message-----
> From: Rajani Karuturi [mailto:rajani@apache.org]
> Sent: Tuesday, April 12, 2016 6:15 AM
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack 4.7 password reset issue.
>
> Can you check the vm_instance table for the VR entry and update
> vm_template_id?
>
> This might be helpful
> https://gist.github.com/terbolous/102ae8edd1cda192561c
>
> ~Rajani
>
> On Sat, Apr 9, 2016 at 5:45 AM, Richard Klein (RSI) <rk...@rsitex.com>
> wrote:
>
> > I found the password reset issue and it ended up being a Python script
> > on the VR. I ended up modifying the "/opt/cloud/bin/configure.py" to
> > resolve the issue. Basically there is a "/etc/cloud/vmpassword.json"
> > file that is updated with the IP/password pair when the GUI password
> > change is performed. During the power on process the VM configuration
> > info is sent to the router which reads the vmpassword.json file and
> > sends the password changes to the password server cache file. When
> > the client retrieved the password it was cleared from the password
> > cache file but not the vmpassword.json file. So every time a VM
> > started the last password reset was sent to the password server again.
> >
> > The question I have now is how do I get the system VM template updated
> > with the change? Since we are using CS v4.7 we used the system
> > template for v4.6 per the installation instructions for CentOS7 and
> > KVM. I performed the following steps to use a new system VM template:
> >
> > * I copied the system vm template QCOW2 file from secondary storage to
> > a work server and made a backup of it.
> > * On the work server I mounted the QCOW2 template file using
> "guestmount"
> > tools and made the code changes to the template.
> > * I then copied this modified template file to a web server and
> > registered the template in cloudstack with all checkboxes off except for
> "routing".
> > * Then we set the cloudstack global value of "router.template.kvm" to
> > the name of the new template.
> > * The management services were restarted.
> > * I picked a test VR, powered it off, destroyed it then let the system
> > recreate it.
> > * When I look at the code I changed on the new VR it does not appear.
> >
> > I even doubled checked the database and the vm_instance table for the
> > test VR showed the new template ID. I must be missing something or I
> > don't really understand how the system templates are created. Any
> > help/suggestions would be appreciated.
> >
> >
> >
> > Richard Klein <rk...@rsitex.com>
> > RSI
> > 5426 Guadalupe, Suite 100
> > Austin TX 78751
> >
> >
> >
> > > -----Original Message-----
> > > From: Richard Klein (RSI)
> > > Sent: Tuesday, April 05, 2016 2:32 PM
> > > To: users@cloudstack.apache.org
> > > Subject: RE: Cloudstack 4.7 password reset issue.
> > >
> > > The snippets for before and after the reboot via console look the
> > > same
> > so I
> > > pasted the 2nd set of message instead of the first. Sorry about that.
> > I did
> > > discover that the /var/lib/dhclient/dhclient.leases existed but was
> > empty. I've
> > > run across an issue with CentOS 7 where the lease file is missing so
> > > I
> > wrote a
> > > "cloud-dhcp-check" service that makes sure it exists but now I need
> > > to
> > validate
> > > its content. That being said, I have insured that the
> > > dhclient_leases
> > was valid
> > > and replicated the problem.
> > >
> > > The cloud-set-guest-xxxx scripts are from the master branch GitHub
> > repository
> > > for apaches/cloudstack using the
> > > "
> > https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-se
> > t-
> > > guest-password.in" and the
> > > "
> > https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-se
> > t-
> > > guest-sshkey.in" links.
> > >
> > > I have attached the entire log from the VR but have some snippets
> > > below
> > along
> > > with the VM client logs and the issue still occurs after fixing the
> > > dhcp
> > lease file.
> > > I did not perform any password resets via the GUI during this process.
> > >
> >
> >
Re: Cloudstack 4.7 password reset issue - resolved.
Posted by Rafael Weingärtner <ra...@gmail.com>.
Those are great improvements, please create the PRs that we will review and
help you prepare the code to be merged. Just be aware that we are going to
have a feature freeze starting next Monday (if I am not mistaken) until the
4.9 version is closed.
On Fri, Apr 15, 2016 at 3:12 PM, Richard Klein (RSI) <rk...@rsitex.com>
wrote:
> Thanks for the replay and I like the 3 PR as well. The changes are
> related to the environment we are using. We are using CS 4.7.0 on CentOS7
> and VXLAN with a two 10Mb NICs per host that is bonded via multi-stack LACP
> switch for performance and redundancy. We ran into the following 3 issues
> with this configuration.
>
> 1) The bond0 interface had 2 bridge interfaces. The "cloudbr0" is bridged
> to bond0. It's used for management and VLAN related guest networks. We
> were limited to 1500 MTU in order to be compatible with our infrastructure
> switches. The "cloudbr1" is bridge to "bond0.xxxx" where "xxxx" is an
> internal VLAN ID that is restricted to Gluster and VXLAN traffic as well as
> using jumbo frames. The problem was with the logic in the "modifyvxlan.sh"
> script. When creating a VXLAN isolated network it would always use the
> "cloudbr0" interface even though the traffic label was defined as
> "cloudbr1". This was due to how the physical interface search was being
> performed. There was even a "TODO" in the script that suggesting passing
> the traffic label instead of doing a search. So I modified the Java back
> end code and the "modifyvslan.sh" script to pass the traffic label and this
> corrected the problem and used the "cloudbr1" interface.
>
> 2) We discovered a very odd problem when migrating instances that use
> VXLAN. For some reason we always got an error like "brbond0" not found. I
> can look up the exact verbiage but basically it was not able to create the
> proper VXLAN bridges and such on the target host because it couldn't find
> the physical interface of "brbond0" which is true. The bridge interface
> should have been "brvx-xxxxx". This turned out to be a problem with
> libvirt and not cloudstack. You could replicate the problem completely
> isolated from the Cloudstack environment. For some reason I still can't
> explain, libvirt would always fail migrating a VXLAN. Oddly enough this
> only happened if the VXLAN bridge interface created by CS started with
> "brvx-xxxx" (-xxxx is the VXLAN ID). Even if manually created and migrated
> (outside of CS control) we found that this would happen so it behaved as
> bug in libvirt not CS. The solution was quite simple. We change the code
> that builds the VXLAN bridges to use "vxbr-xxxx". I really can't explain
> why but the name changed resolved the issue.
>
> 3) This last one is the password change issue we've been discussing which
> as you know was resolved by changing the "configure.py" script.
>
> I will try a pull request on number 3 first. This is the simplest way for
> me to get use to the process. I will then follow up on the others as time
> permits. We are about to implement CS in a production environment so
> hopefully I can get to the PR soon.
>
>
> Richard Klein <rk...@rsitex.com>
> RSI
> 5426 Guadalupe, Suite 100
> Austin TX 78751
>
>
>
>
>
> > -----Original Message-----
> > From: Rafael Weingärtner [mailto:rafaelweingartner@gmail.com]
> > Sent: Friday, April 15, 2016 12:02 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Cloudstack 4.7 password reset issue - resolved.
> >
> > Would it be possible for you to explain a little bit these changes?
> >
> > I believe a PR per change would be the best way to go.
> >
> > On Fri, Apr 15, 2016 at 1:31 PM, Richard Klein (RSI) <rk...@rsitex.com>
> > wrote:
> >
> > > I would be happy to submit a pull request but I am relatively new to
> > > using Git and GitHub. I have a lot of experience with SVN and CVS. I
> > > have read the following link about the process:
> > >
> > > * https://cloudstack.apache.org/developers.html
> > > * https://help.github.com/articles/creating-a-pull-request/
> > >
> > > I have forked the apache/cloudstack on GitHub and have been making
> > > changes to the 4.7.0 version on a separate branch. This branch
> > > contains several code changes we had to make in order to get CS to run
> in
> > our environment.
> > > Since I am not familiar with Mavin I have created some non-standard
> > > version numbers in order to distinguish the RPMs and use a private
> > > repository so we can control the upgrade process.
> > >
> > > I see 2 options on submitting a pull request. First is to submit it
> > > from the existing branch that contains all the modified code we've
> > > made to 4.7.0. The only downside is it contains a lot of "pom.xml"
> > > version number changes as well. The Second option is to create a
> > > branch for each of the 3 types of fixes we have made and do a pull
> request
> > for each one.
> > >
> > > Let me know if there are any additional resources I need to read up on
> > > and the proper method of submitting a pull request.
> > >
> > > Thanks!
> > >
> > > Richard Klein <rk...@rsitex.com>
> > > RSI
> > > 5426 Guadalupe, Suite 100
> > > Austin TX 78751
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Remi Bergsma [mailto:RBergsma@schubergphilis.com]
> > > > Sent: Thursday, April 14, 2016 3:35 PM
> > > > To: users@cloudstack.apache.org
> > > > Subject: Re: Cloudstack 4.7 password reset issue - resolved.
> > > >
> > > > Hi Richard,
> > > >
> > > > Great you fixed it! Can you send the patch of your fix as a spul
> > > > request
> > > on
> > > > github?
> > > >
> > > > Required upgrade is yes when the router reports a version older than
> > > > the minreq.sysvm.version (or similar) global setting. It's used to
> > > > upgrade
> > > systemvm
> > > > templates.
> > > >
> > > > Regards, Remi
> > > >
> > > > Sent from my iPhone
> > > >
> > > > > On 13 Apr 2016, at 22:23, Richard Klein (RSI) <rk...@rsitex.com>
> > > wrote:
> > > > >
> > > > > I finally found the problem and resolved the issue. The problem
> > > > > was
> > > in the
> > > > Python code change I made. I had a flag variable that indicated to
> > > > save
> > > data
> > > > when it was changed while processing a list. This worked fine as
> > > > long
> > > as it
> > > > executed the logic and defined the flag variable. The problem was
> > > > during startup when it doesn't go through the loop and the flag
> > > > variable was undefined. This cause the "update_config.py" to fail
> > > > which then bubbled
> > > back
> > > > up as an error and prevent the router from starting.
> > > > >
> > > > > Once I changed the code and rebuilt the project all worked well
> > > > > and
> > > the bug
> > > > is fixed. Thanks so much for everyone's help. This process was
> > > > very educational and looking forward to learning more.
> > > > >
> > > > > I do have one question just out of curiosity. What makes the
> > > > > "Requires
> > > > Upgrade" column on the Home->Infrastructure->Virtual Router page
> > > indicated
> > > > "Yes"?
> > > > >
> > > > > Thanks again,
> > > > >
> > > > >
> > > > > Richard Klein <rk...@rsitex.com> RSI
> > > > > 5426 Guadalupe, Suite 100
> > > > > Austin TX 78751
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Rajani Karuturi [mailto:rajani@apache.org]
> > > > >> Sent: Tuesday, April 12, 2016 6:15 AM
> > > > >> To: users@cloudstack.apache.org
> > > > >> Subject: Re: Cloudstack 4.7 password reset issue.
> > > > >>
> > > > >> Can you check the vm_instance table for the VR entry and update
> > > > >> vm_template_id?
> > > > >>
> > > > >> This might be helpful
> > > > >> https://gist.github.com/terbolous/102ae8edd1cda192561c
> > > > >>
> > > > >> ~Rajani
> > > > >>
> > > > >> On Sat, Apr 9, 2016 at 5:45 AM, Richard Klein (RSI)
> > > > >> <rk...@rsitex.com>
> > > > >> wrote:
> > > > >>
> > > > >>> I found the password reset issue and it ended up being a Python
> > > > >>> script on the VR. I ended up modifying the
> > > > >>> "/opt/cloud/bin/configure.py" to resolve the issue. Basically
> there
> > > is a
> > > > "/etc/cloud/vmpassword.json"
> > > > >>> file that is updated with the IP/password pair when the GUI
> password
> > > > >>> change is performed. During the power on process the VM
> > > > >>> configuration info is sent to the router which reads the
> > > > >>> vmpassword.json file and sends the password changes to the
> password
> > > > >>> server cache file. When the client retrieved the password it was
> > > > >>> cleared from the password cache file but not the vmpassword.json
> > > > >>> file. So every time a VM started the last password reset was
> sent
> > > to the
> > > > password server again.
> > > > >>>
> > > > >>> The question I have now is how do I get the system VM template
> > > > >>> updated with the change? Since we are using CS v4.7 we used the
> > > > >>> system template for v4.6 per the installation instructions for
> > > > >>> CentOS7 and KVM. I performed the following steps to use a new
> > system
> > > > VM template:
> > > > >>>
> > > > >>> * I copied the system vm template QCOW2 file from secondary
> storage
> > > > >>> to a work server and made a backup of it.
> > > > >>> * On the work server I mounted the QCOW2 template file using
> > > > >> "guestmount"
> > > > >>> tools and made the code changes to the template.
> > > > >>> * I then copied this modified template file to a web server and
> > > > >>> registered the template in cloudstack with all checkboxes off
> except
> > > > >>> for
> > > > >> "routing".
> > > > >>> * Then we set the cloudstack global value of
> "router.template.kvm"
> > > > >>> to the name of the new template.
> > > > >>> * The management services were restarted.
> > > > >>> * I picked a test VR, powered it off, destroyed it then let the
> > > > >>> system recreate it.
> > > > >>> * When I look at the code I changed on the new VR it does not
> appear.
> > > > >>>
> > > > >>> I even doubled checked the database and the vm_instance table for
> > > > >>> the test VR showed the new template ID. I must be missing
> something
> > > > >>> or I don't really understand how the system templates are
> created.
> > > > >>> Any help/suggestions would be appreciated.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> Richard Klein <rk...@rsitex.com>
> > > > >>> RSI
> > > > >>> 5426 Guadalupe, Suite 100
> > > > >>> Austin TX 78751
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>> -----Original Message-----
> > > > >>>> From: Richard Klein (RSI)
> > > > >>>> Sent: Tuesday, April 05, 2016 2:32 PM
> > > > >>>> To: users@cloudstack.apache.org
> > > > >>>> Subject: RE: Cloudstack 4.7 password reset issue.
> > > > >>>>
> > > > >>>> The snippets for before and after the reboot via console look
> the
> > > > >>>> same
> > > > >>> so I
> > > > >>>> pasted the 2nd set of message instead of the first. Sorry about
> > > that.
> > > > >>> I did
> > > > >>>> discover that the /var/lib/dhclient/dhclient.leases existed but
> was
> > > > >>> empty. I've
> > > > >>>> run across an issue with CentOS 7 where the lease file is
> missing
> > > > >>>> so I
> > > > >>> wrote a
> > > > >>>> "cloud-dhcp-check" service that makes sure it exists but now I
> need
> > > > >>>> to
> > > > >>> validate
> > > > >>>> its content. That being said, I have insured that the
> > > > >>>> dhclient_leases
> > > > >>> was valid
> > > > >>>> and replicated the problem.
> > > > >>>>
> > > > >>>> The cloud-set-guest-xxxx scripts are from the master branch
> GitHub
> > > > >>> repository
> > > > >>>> for apaches/cloudstack using the
> > > > >>>> "
> > > > >>>
> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-
> > > > >>> se
> > > > >>> t-
> > > > >>>> guest-password.in" and the
> > > > >>>> "
> > > > >>>
> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-
> > > > >>> se
> > > > >>> t-
> > > > >>>> guest-sshkey.in" links.
> > > > >>>>
> > > > >>>> I have attached the entire log from the VR but have some
> snippets
> > > > >>>> below
> > > > >>> along
> > > > >>>> with the VM client logs and the issue still occurs after fixing
> the
> > > > >>>> dhcp
> > > > >>> lease file.
> > > > >>>> I did not perform any password resets via the GUI during this
> > > process.
> > > > >>>
> > > > >>>
> > >
> >
> >
> >
> > --
> > Rafael Weingärtner
>
--
Rafael Weingärtner
RE: Cloudstack 4.7 password reset issue - resolved.
Posted by "Richard Klein (RSI)" <rk...@rsitex.com>.
Thanks for the replay and I like the 3 PR as well. The changes are related to the environment we are using. We are using CS 4.7.0 on CentOS7 and VXLAN with a two 10Mb NICs per host that is bonded via multi-stack LACP switch for performance and redundancy. We ran into the following 3 issues with this configuration.
1) The bond0 interface had 2 bridge interfaces. The "cloudbr0" is bridged to bond0. It's used for management and VLAN related guest networks. We were limited to 1500 MTU in order to be compatible with our infrastructure switches. The "cloudbr1" is bridge to "bond0.xxxx" where "xxxx" is an internal VLAN ID that is restricted to Gluster and VXLAN traffic as well as using jumbo frames. The problem was with the logic in the "modifyvxlan.sh" script. When creating a VXLAN isolated network it would always use the "cloudbr0" interface even though the traffic label was defined as "cloudbr1". This was due to how the physical interface search was being performed. There was even a "TODO" in the script that suggesting passing the traffic label instead of doing a search. So I modified the Java back end code and the "modifyvslan.sh" script to pass the traffic label and this corrected the problem and used the "cloudbr1" interface.
2) We discovered a very odd problem when migrating instances that use VXLAN. For some reason we always got an error like "brbond0" not found. I can look up the exact verbiage but basically it was not able to create the proper VXLAN bridges and such on the target host because it couldn't find the physical interface of "brbond0" which is true. The bridge interface should have been "brvx-xxxxx". This turned out to be a problem with libvirt and not cloudstack. You could replicate the problem completely isolated from the Cloudstack environment. For some reason I still can't explain, libvirt would always fail migrating a VXLAN. Oddly enough this only happened if the VXLAN bridge interface created by CS started with "brvx-xxxx" (-xxxx is the VXLAN ID). Even if manually created and migrated (outside of CS control) we found that this would happen so it behaved as bug in libvirt not CS. The solution was quite simple. We change the code that builds the VXLAN bridges to use "vxbr-xxxx". I really can't explain why but the name changed resolved the issue.
3) This last one is the password change issue we've been discussing which as you know was resolved by changing the "configure.py" script.
I will try a pull request on number 3 first. This is the simplest way for me to get use to the process. I will then follow up on the others as time permits. We are about to implement CS in a production environment so hopefully I can get to the PR soon.
Richard Klein <rk...@rsitex.com>
RSI
5426 Guadalupe, Suite 100
Austin TX 78751
> -----Original Message-----
> From: Rafael Weingärtner [mailto:rafaelweingartner@gmail.com]
> Sent: Friday, April 15, 2016 12:02 PM
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack 4.7 password reset issue - resolved.
>
> Would it be possible for you to explain a little bit these changes?
>
> I believe a PR per change would be the best way to go.
>
> On Fri, Apr 15, 2016 at 1:31 PM, Richard Klein (RSI) <rk...@rsitex.com>
> wrote:
>
> > I would be happy to submit a pull request but I am relatively new to
> > using Git and GitHub. I have a lot of experience with SVN and CVS. I
> > have read the following link about the process:
> >
> > * https://cloudstack.apache.org/developers.html
> > * https://help.github.com/articles/creating-a-pull-request/
> >
> > I have forked the apache/cloudstack on GitHub and have been making
> > changes to the 4.7.0 version on a separate branch. This branch
> > contains several code changes we had to make in order to get CS to run in
> our environment.
> > Since I am not familiar with Mavin I have created some non-standard
> > version numbers in order to distinguish the RPMs and use a private
> > repository so we can control the upgrade process.
> >
> > I see 2 options on submitting a pull request. First is to submit it
> > from the existing branch that contains all the modified code we've
> > made to 4.7.0. The only downside is it contains a lot of "pom.xml"
> > version number changes as well. The Second option is to create a
> > branch for each of the 3 types of fixes we have made and do a pull request
> for each one.
> >
> > Let me know if there are any additional resources I need to read up on
> > and the proper method of submitting a pull request.
> >
> > Thanks!
> >
> > Richard Klein <rk...@rsitex.com>
> > RSI
> > 5426 Guadalupe, Suite 100
> > Austin TX 78751
> >
> >
> >
> > > -----Original Message-----
> > > From: Remi Bergsma [mailto:RBergsma@schubergphilis.com]
> > > Sent: Thursday, April 14, 2016 3:35 PM
> > > To: users@cloudstack.apache.org
> > > Subject: Re: Cloudstack 4.7 password reset issue - resolved.
> > >
> > > Hi Richard,
> > >
> > > Great you fixed it! Can you send the patch of your fix as a spul
> > > request
> > on
> > > github?
> > >
> > > Required upgrade is yes when the router reports a version older than
> > > the minreq.sysvm.version (or similar) global setting. It's used to
> > > upgrade
> > systemvm
> > > templates.
> > >
> > > Regards, Remi
> > >
> > > Sent from my iPhone
> > >
> > > > On 13 Apr 2016, at 22:23, Richard Klein (RSI) <rk...@rsitex.com>
> > wrote:
> > > >
> > > > I finally found the problem and resolved the issue. The problem
> > > > was
> > in the
> > > Python code change I made. I had a flag variable that indicated to
> > > save
> > data
> > > when it was changed while processing a list. This worked fine as
> > > long
> > as it
> > > executed the logic and defined the flag variable. The problem was
> > > during startup when it doesn't go through the loop and the flag
> > > variable was undefined. This cause the "update_config.py" to fail
> > > which then bubbled
> > back
> > > up as an error and prevent the router from starting.
> > > >
> > > > Once I changed the code and rebuilt the project all worked well
> > > > and
> > the bug
> > > is fixed. Thanks so much for everyone's help. This process was
> > > very educational and looking forward to learning more.
> > > >
> > > > I do have one question just out of curiosity. What makes the
> > > > "Requires
> > > Upgrade" column on the Home->Infrastructure->Virtual Router page
> > indicated
> > > "Yes"?
> > > >
> > > > Thanks again,
> > > >
> > > >
> > > > Richard Klein <rk...@rsitex.com> RSI
> > > > 5426 Guadalupe, Suite 100
> > > > Austin TX 78751
> > > >
> > > >
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Rajani Karuturi [mailto:rajani@apache.org]
> > > >> Sent: Tuesday, April 12, 2016 6:15 AM
> > > >> To: users@cloudstack.apache.org
> > > >> Subject: Re: Cloudstack 4.7 password reset issue.
> > > >>
> > > >> Can you check the vm_instance table for the VR entry and update
> > > >> vm_template_id?
> > > >>
> > > >> This might be helpful
> > > >> https://gist.github.com/terbolous/102ae8edd1cda192561c
> > > >>
> > > >> ~Rajani
> > > >>
> > > >> On Sat, Apr 9, 2016 at 5:45 AM, Richard Klein (RSI)
> > > >> <rk...@rsitex.com>
> > > >> wrote:
> > > >>
> > > >>> I found the password reset issue and it ended up being a Python
> > > >>> script on the VR. I ended up modifying the
> > > >>> "/opt/cloud/bin/configure.py" to resolve the issue. Basically there
> > is a
> > > "/etc/cloud/vmpassword.json"
> > > >>> file that is updated with the IP/password pair when the GUI password
> > > >>> change is performed. During the power on process the VM
> > > >>> configuration info is sent to the router which reads the
> > > >>> vmpassword.json file and sends the password changes to the password
> > > >>> server cache file. When the client retrieved the password it was
> > > >>> cleared from the password cache file but not the vmpassword.json
> > > >>> file. So every time a VM started the last password reset was sent
> > to the
> > > password server again.
> > > >>>
> > > >>> The question I have now is how do I get the system VM template
> > > >>> updated with the change? Since we are using CS v4.7 we used the
> > > >>> system template for v4.6 per the installation instructions for
> > > >>> CentOS7 and KVM. I performed the following steps to use a new
> system
> > > VM template:
> > > >>>
> > > >>> * I copied the system vm template QCOW2 file from secondary storage
> > > >>> to a work server and made a backup of it.
> > > >>> * On the work server I mounted the QCOW2 template file using
> > > >> "guestmount"
> > > >>> tools and made the code changes to the template.
> > > >>> * I then copied this modified template file to a web server and
> > > >>> registered the template in cloudstack with all checkboxes off except
> > > >>> for
> > > >> "routing".
> > > >>> * Then we set the cloudstack global value of "router.template.kvm"
> > > >>> to the name of the new template.
> > > >>> * The management services were restarted.
> > > >>> * I picked a test VR, powered it off, destroyed it then let the
> > > >>> system recreate it.
> > > >>> * When I look at the code I changed on the new VR it does not appear.
> > > >>>
> > > >>> I even doubled checked the database and the vm_instance table for
> > > >>> the test VR showed the new template ID. I must be missing something
> > > >>> or I don't really understand how the system templates are created.
> > > >>> Any help/suggestions would be appreciated.
> > > >>>
> > > >>>
> > > >>>
> > > >>> Richard Klein <rk...@rsitex.com>
> > > >>> RSI
> > > >>> 5426 Guadalupe, Suite 100
> > > >>> Austin TX 78751
> > > >>>
> > > >>>
> > > >>>
> > > >>>> -----Original Message-----
> > > >>>> From: Richard Klein (RSI)
> > > >>>> Sent: Tuesday, April 05, 2016 2:32 PM
> > > >>>> To: users@cloudstack.apache.org
> > > >>>> Subject: RE: Cloudstack 4.7 password reset issue.
> > > >>>>
> > > >>>> The snippets for before and after the reboot via console look the
> > > >>>> same
> > > >>> so I
> > > >>>> pasted the 2nd set of message instead of the first. Sorry about
> > that.
> > > >>> I did
> > > >>>> discover that the /var/lib/dhclient/dhclient.leases existed but was
> > > >>> empty. I've
> > > >>>> run across an issue with CentOS 7 where the lease file is missing
> > > >>>> so I
> > > >>> wrote a
> > > >>>> "cloud-dhcp-check" service that makes sure it exists but now I need
> > > >>>> to
> > > >>> validate
> > > >>>> its content. That being said, I have insured that the
> > > >>>> dhclient_leases
> > > >>> was valid
> > > >>>> and replicated the problem.
> > > >>>>
> > > >>>> The cloud-set-guest-xxxx scripts are from the master branch GitHub
> > > >>> repository
> > > >>>> for apaches/cloudstack using the
> > > >>>> "
> > > >>> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-
> > > >>> se
> > > >>> t-
> > > >>>> guest-password.in" and the
> > > >>>> "
> > > >>> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-
> > > >>> se
> > > >>> t-
> > > >>>> guest-sshkey.in" links.
> > > >>>>
> > > >>>> I have attached the entire log from the VR but have some snippets
> > > >>>> below
> > > >>> along
> > > >>>> with the VM client logs and the issue still occurs after fixing the
> > > >>>> dhcp
> > > >>> lease file.
> > > >>>> I did not perform any password resets via the GUI during this
> > process.
> > > >>>
> > > >>>
> >
>
>
>
> --
> Rafael Weingärtner
Re: Cloudstack 4.7 password reset issue - resolved.
Posted by Rafael Weingärtner <ra...@gmail.com>.
Would it be possible for you to explain a little bit these changes?
I believe a PR per change would be the best way to go.
On Fri, Apr 15, 2016 at 1:31 PM, Richard Klein (RSI) <rk...@rsitex.com>
wrote:
> I would be happy to submit a pull request but I am relatively new to using
> Git and GitHub. I have a lot of experience with SVN and CVS. I have read
> the following link about the process:
>
> * https://cloudstack.apache.org/developers.html
> * https://help.github.com/articles/creating-a-pull-request/
>
> I have forked the apache/cloudstack on GitHub and have been making changes
> to the 4.7.0 version on a separate branch. This branch contains several
> code changes we had to make in order to get CS to run in our environment.
> Since I am not familiar with Mavin I have created some non-standard version
> numbers in order to distinguish the RPMs and use a private repository so we
> can control the upgrade process.
>
> I see 2 options on submitting a pull request. First is to submit it from
> the existing branch that contains all the modified code we've made to
> 4.7.0. The only downside is it contains a lot of "pom.xml" version number
> changes as well. The Second option is to create a branch for each of the 3
> types of fixes we have made and do a pull request for each one.
>
> Let me know if there are any additional resources I need to read up on and
> the proper method of submitting a pull request.
>
> Thanks!
>
> Richard Klein <rk...@rsitex.com>
> RSI
> 5426 Guadalupe, Suite 100
> Austin TX 78751
>
>
>
> > -----Original Message-----
> > From: Remi Bergsma [mailto:RBergsma@schubergphilis.com]
> > Sent: Thursday, April 14, 2016 3:35 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Cloudstack 4.7 password reset issue - resolved.
> >
> > Hi Richard,
> >
> > Great you fixed it! Can you send the patch of your fix as a spul request
> on
> > github?
> >
> > Required upgrade is yes when the router reports a version older than the
> > minreq.sysvm.version (or similar) global setting. It's used to upgrade
> systemvm
> > templates.
> >
> > Regards, Remi
> >
> > Sent from my iPhone
> >
> > > On 13 Apr 2016, at 22:23, Richard Klein (RSI) <rk...@rsitex.com>
> wrote:
> > >
> > > I finally found the problem and resolved the issue. The problem was
> in the
> > Python code change I made. I had a flag variable that indicated to save
> data
> > when it was changed while processing a list. This worked fine as long
> as it
> > executed the logic and defined the flag variable. The problem was during
> > startup when it doesn't go through the loop and the flag variable was
> > undefined. This cause the "update_config.py" to fail which then bubbled
> back
> > up as an error and prevent the router from starting.
> > >
> > > Once I changed the code and rebuilt the project all worked well and
> the bug
> > is fixed. Thanks so much for everyone's help. This process was very
> > educational and looking forward to learning more.
> > >
> > > I do have one question just out of curiosity. What makes the "Requires
> > Upgrade" column on the Home->Infrastructure->Virtual Router page
> indicated
> > "Yes"?
> > >
> > > Thanks again,
> > >
> > >
> > > Richard Klein <rk...@rsitex.com>
> > > RSI
> > > 5426 Guadalupe, Suite 100
> > > Austin TX 78751
> > >
> > >
> > >
> > >
> > >> -----Original Message-----
> > >> From: Rajani Karuturi [mailto:rajani@apache.org]
> > >> Sent: Tuesday, April 12, 2016 6:15 AM
> > >> To: users@cloudstack.apache.org
> > >> Subject: Re: Cloudstack 4.7 password reset issue.
> > >>
> > >> Can you check the vm_instance table for the VR entry and update
> > >> vm_template_id?
> > >>
> > >> This might be helpful
> > >> https://gist.github.com/terbolous/102ae8edd1cda192561c
> > >>
> > >> ~Rajani
> > >>
> > >> On Sat, Apr 9, 2016 at 5:45 AM, Richard Klein (RSI)
> > >> <rk...@rsitex.com>
> > >> wrote:
> > >>
> > >>> I found the password reset issue and it ended up being a Python
> > >>> script on the VR. I ended up modifying the
> > >>> "/opt/cloud/bin/configure.py" to resolve the issue. Basically there
> is a
> > "/etc/cloud/vmpassword.json"
> > >>> file that is updated with the IP/password pair when the GUI password
> > >>> change is performed. During the power on process the VM
> > >>> configuration info is sent to the router which reads the
> > >>> vmpassword.json file and sends the password changes to the password
> > >>> server cache file. When the client retrieved the password it was
> > >>> cleared from the password cache file but not the vmpassword.json
> > >>> file. So every time a VM started the last password reset was sent
> to the
> > password server again.
> > >>>
> > >>> The question I have now is how do I get the system VM template
> > >>> updated with the change? Since we are using CS v4.7 we used the
> > >>> system template for v4.6 per the installation instructions for
> > >>> CentOS7 and KVM. I performed the following steps to use a new system
> > VM template:
> > >>>
> > >>> * I copied the system vm template QCOW2 file from secondary storage
> > >>> to a work server and made a backup of it.
> > >>> * On the work server I mounted the QCOW2 template file using
> > >> "guestmount"
> > >>> tools and made the code changes to the template.
> > >>> * I then copied this modified template file to a web server and
> > >>> registered the template in cloudstack with all checkboxes off except
> > >>> for
> > >> "routing".
> > >>> * Then we set the cloudstack global value of "router.template.kvm"
> > >>> to the name of the new template.
> > >>> * The management services were restarted.
> > >>> * I picked a test VR, powered it off, destroyed it then let the
> > >>> system recreate it.
> > >>> * When I look at the code I changed on the new VR it does not appear.
> > >>>
> > >>> I even doubled checked the database and the vm_instance table for
> > >>> the test VR showed the new template ID. I must be missing something
> > >>> or I don't really understand how the system templates are created.
> > >>> Any help/suggestions would be appreciated.
> > >>>
> > >>>
> > >>>
> > >>> Richard Klein <rk...@rsitex.com>
> > >>> RSI
> > >>> 5426 Guadalupe, Suite 100
> > >>> Austin TX 78751
> > >>>
> > >>>
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: Richard Klein (RSI)
> > >>>> Sent: Tuesday, April 05, 2016 2:32 PM
> > >>>> To: users@cloudstack.apache.org
> > >>>> Subject: RE: Cloudstack 4.7 password reset issue.
> > >>>>
> > >>>> The snippets for before and after the reboot via console look the
> > >>>> same
> > >>> so I
> > >>>> pasted the 2nd set of message instead of the first. Sorry about
> that.
> > >>> I did
> > >>>> discover that the /var/lib/dhclient/dhclient.leases existed but was
> > >>> empty. I've
> > >>>> run across an issue with CentOS 7 where the lease file is missing
> > >>>> so I
> > >>> wrote a
> > >>>> "cloud-dhcp-check" service that makes sure it exists but now I need
> > >>>> to
> > >>> validate
> > >>>> its content. That being said, I have insured that the
> > >>>> dhclient_leases
> > >>> was valid
> > >>>> and replicated the problem.
> > >>>>
> > >>>> The cloud-set-guest-xxxx scripts are from the master branch GitHub
> > >>> repository
> > >>>> for apaches/cloudstack using the
> > >>>> "
> > >>> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-
> > >>> se
> > >>> t-
> > >>>> guest-password.in" and the
> > >>>> "
> > >>> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-
> > >>> se
> > >>> t-
> > >>>> guest-sshkey.in" links.
> > >>>>
> > >>>> I have attached the entire log from the VR but have some snippets
> > >>>> below
> > >>> along
> > >>>> with the VM client logs and the issue still occurs after fixing the
> > >>>> dhcp
> > >>> lease file.
> > >>>> I did not perform any password resets via the GUI during this
> process.
> > >>>
> > >>>
>
--
Rafael Weingärtner
RE: Cloudstack 4.7 password reset issue - resolved.
Posted by "Richard Klein (RSI)" <rk...@rsitex.com>.
I would be happy to submit a pull request but I am relatively new to using Git and GitHub. I have a lot of experience with SVN and CVS. I have read the following link about the process:
* https://cloudstack.apache.org/developers.html
* https://help.github.com/articles/creating-a-pull-request/
I have forked the apache/cloudstack on GitHub and have been making changes to the 4.7.0 version on a separate branch. This branch contains several code changes we had to make in order to get CS to run in our environment. Since I am not familiar with Mavin I have created some non-standard version numbers in order to distinguish the RPMs and use a private repository so we can control the upgrade process.
I see 2 options on submitting a pull request. First is to submit it from the existing branch that contains all the modified code we've made to 4.7.0. The only downside is it contains a lot of "pom.xml" version number changes as well. The Second option is to create a branch for each of the 3 types of fixes we have made and do a pull request for each one.
Let me know if there are any additional resources I need to read up on and the proper method of submitting a pull request.
Thanks!
Richard Klein <rk...@rsitex.com>
RSI
5426 Guadalupe, Suite 100
Austin TX 78751
> -----Original Message-----
> From: Remi Bergsma [mailto:RBergsma@schubergphilis.com]
> Sent: Thursday, April 14, 2016 3:35 PM
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack 4.7 password reset issue - resolved.
>
> Hi Richard,
>
> Great you fixed it! Can you send the patch of your fix as a spul request on
> github?
>
> Required upgrade is yes when the router reports a version older than the
> minreq.sysvm.version (or similar) global setting. It's used to upgrade systemvm
> templates.
>
> Regards, Remi
>
> Sent from my iPhone
>
> > On 13 Apr 2016, at 22:23, Richard Klein (RSI) <rk...@rsitex.com> wrote:
> >
> > I finally found the problem and resolved the issue. The problem was in the
> Python code change I made. I had a flag variable that indicated to save data
> when it was changed while processing a list. This worked fine as long as it
> executed the logic and defined the flag variable. The problem was during
> startup when it doesn't go through the loop and the flag variable was
> undefined. This cause the "update_config.py" to fail which then bubbled back
> up as an error and prevent the router from starting.
> >
> > Once I changed the code and rebuilt the project all worked well and the bug
> is fixed. Thanks so much for everyone's help. This process was very
> educational and looking forward to learning more.
> >
> > I do have one question just out of curiosity. What makes the "Requires
> Upgrade" column on the Home->Infrastructure->Virtual Router page indicated
> "Yes"?
> >
> > Thanks again,
> >
> >
> > Richard Klein <rk...@rsitex.com>
> > RSI
> > 5426 Guadalupe, Suite 100
> > Austin TX 78751
> >
> >
> >
> >
> >> -----Original Message-----
> >> From: Rajani Karuturi [mailto:rajani@apache.org]
> >> Sent: Tuesday, April 12, 2016 6:15 AM
> >> To: users@cloudstack.apache.org
> >> Subject: Re: Cloudstack 4.7 password reset issue.
> >>
> >> Can you check the vm_instance table for the VR entry and update
> >> vm_template_id?
> >>
> >> This might be helpful
> >> https://gist.github.com/terbolous/102ae8edd1cda192561c
> >>
> >> ~Rajani
> >>
> >> On Sat, Apr 9, 2016 at 5:45 AM, Richard Klein (RSI)
> >> <rk...@rsitex.com>
> >> wrote:
> >>
> >>> I found the password reset issue and it ended up being a Python
> >>> script on the VR. I ended up modifying the
> >>> "/opt/cloud/bin/configure.py" to resolve the issue. Basically there is a
> "/etc/cloud/vmpassword.json"
> >>> file that is updated with the IP/password pair when the GUI password
> >>> change is performed. During the power on process the VM
> >>> configuration info is sent to the router which reads the
> >>> vmpassword.json file and sends the password changes to the password
> >>> server cache file. When the client retrieved the password it was
> >>> cleared from the password cache file but not the vmpassword.json
> >>> file. So every time a VM started the last password reset was sent to the
> password server again.
> >>>
> >>> The question I have now is how do I get the system VM template
> >>> updated with the change? Since we are using CS v4.7 we used the
> >>> system template for v4.6 per the installation instructions for
> >>> CentOS7 and KVM. I performed the following steps to use a new system
> VM template:
> >>>
> >>> * I copied the system vm template QCOW2 file from secondary storage
> >>> to a work server and made a backup of it.
> >>> * On the work server I mounted the QCOW2 template file using
> >> "guestmount"
> >>> tools and made the code changes to the template.
> >>> * I then copied this modified template file to a web server and
> >>> registered the template in cloudstack with all checkboxes off except
> >>> for
> >> "routing".
> >>> * Then we set the cloudstack global value of "router.template.kvm"
> >>> to the name of the new template.
> >>> * The management services were restarted.
> >>> * I picked a test VR, powered it off, destroyed it then let the
> >>> system recreate it.
> >>> * When I look at the code I changed on the new VR it does not appear.
> >>>
> >>> I even doubled checked the database and the vm_instance table for
> >>> the test VR showed the new template ID. I must be missing something
> >>> or I don't really understand how the system templates are created.
> >>> Any help/suggestions would be appreciated.
> >>>
> >>>
> >>>
> >>> Richard Klein <rk...@rsitex.com>
> >>> RSI
> >>> 5426 Guadalupe, Suite 100
> >>> Austin TX 78751
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Richard Klein (RSI)
> >>>> Sent: Tuesday, April 05, 2016 2:32 PM
> >>>> To: users@cloudstack.apache.org
> >>>> Subject: RE: Cloudstack 4.7 password reset issue.
> >>>>
> >>>> The snippets for before and after the reboot via console look the
> >>>> same
> >>> so I
> >>>> pasted the 2nd set of message instead of the first. Sorry about that.
> >>> I did
> >>>> discover that the /var/lib/dhclient/dhclient.leases existed but was
> >>> empty. I've
> >>>> run across an issue with CentOS 7 where the lease file is missing
> >>>> so I
> >>> wrote a
> >>>> "cloud-dhcp-check" service that makes sure it exists but now I need
> >>>> to
> >>> validate
> >>>> its content. That being said, I have insured that the
> >>>> dhclient_leases
> >>> was valid
> >>>> and replicated the problem.
> >>>>
> >>>> The cloud-set-guest-xxxx scripts are from the master branch GitHub
> >>> repository
> >>>> for apaches/cloudstack using the
> >>>> "
> >>> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-
> >>> se
> >>> t-
> >>>> guest-password.in" and the
> >>>> "
> >>> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-
> >>> se
> >>> t-
> >>>> guest-sshkey.in" links.
> >>>>
> >>>> I have attached the entire log from the VR but have some snippets
> >>>> below
> >>> along
> >>>> with the VM client logs and the issue still occurs after fixing the
> >>>> dhcp
> >>> lease file.
> >>>> I did not perform any password resets via the GUI during this process.
> >>>
> >>>
Re: Cloudstack 4.7 password reset issue - resolved.
Posted by Remi Bergsma <RB...@schubergphilis.com>.
Hi Richard,
Great you fixed it! Can you send the patch of your fix as a spul request on github?
Required upgrade is yes when the router reports a version older than the minreq.sysvm.version (or similar) global setting. It's used to upgrade systemvm templates.
Regards, Remi
Sent from my iPhone
> On 13 Apr 2016, at 22:23, Richard Klein (RSI) <rk...@rsitex.com> wrote:
>
> I finally found the problem and resolved the issue. The problem was in the Python code change I made. I had a flag variable that indicated to save data when it was changed while processing a list. This worked fine as long as it executed the logic and defined the flag variable. The problem was during startup when it doesn't go through the loop and the flag variable was undefined. This cause the "update_config.py" to fail which then bubbled back up as an error and prevent the router from starting.
>
> Once I changed the code and rebuilt the project all worked well and the bug is fixed. Thanks so much for everyone's help. This process was very educational and looking forward to learning more.
>
> I do have one question just out of curiosity. What makes the "Requires Upgrade" column on the Home->Infrastructure->Virtual Router page indicated "Yes"?
>
> Thanks again,
>
>
> Richard Klein <rk...@rsitex.com>
> RSI
> 5426 Guadalupe, Suite 100
> Austin TX 78751
>
>
>
>
>> -----Original Message-----
>> From: Rajani Karuturi [mailto:rajani@apache.org]
>> Sent: Tuesday, April 12, 2016 6:15 AM
>> To: users@cloudstack.apache.org
>> Subject: Re: Cloudstack 4.7 password reset issue.
>>
>> Can you check the vm_instance table for the VR entry and update
>> vm_template_id?
>>
>> This might be helpful
>> https://gist.github.com/terbolous/102ae8edd1cda192561c
>>
>> ~Rajani
>>
>> On Sat, Apr 9, 2016 at 5:45 AM, Richard Klein (RSI) <rk...@rsitex.com>
>> wrote:
>>
>>> I found the password reset issue and it ended up being a Python script
>>> on the VR. I ended up modifying the "/opt/cloud/bin/configure.py" to
>>> resolve the issue. Basically there is a "/etc/cloud/vmpassword.json"
>>> file that is updated with the IP/password pair when the GUI password
>>> change is performed. During the power on process the VM configuration
>>> info is sent to the router which reads the vmpassword.json file and
>>> sends the password changes to the password server cache file. When
>>> the client retrieved the password it was cleared from the password
>>> cache file but not the vmpassword.json file. So every time a VM
>>> started the last password reset was sent to the password server again.
>>>
>>> The question I have now is how do I get the system VM template updated
>>> with the change? Since we are using CS v4.7 we used the system
>>> template for v4.6 per the installation instructions for CentOS7 and
>>> KVM. I performed the following steps to use a new system VM template:
>>>
>>> * I copied the system vm template QCOW2 file from secondary storage to
>>> a work server and made a backup of it.
>>> * On the work server I mounted the QCOW2 template file using
>> "guestmount"
>>> tools and made the code changes to the template.
>>> * I then copied this modified template file to a web server and
>>> registered the template in cloudstack with all checkboxes off except for
>> "routing".
>>> * Then we set the cloudstack global value of "router.template.kvm" to
>>> the name of the new template.
>>> * The management services were restarted.
>>> * I picked a test VR, powered it off, destroyed it then let the system
>>> recreate it.
>>> * When I look at the code I changed on the new VR it does not appear.
>>>
>>> I even doubled checked the database and the vm_instance table for the
>>> test VR showed the new template ID. I must be missing something or I
>>> don't really understand how the system templates are created. Any
>>> help/suggestions would be appreciated.
>>>
>>>
>>>
>>> Richard Klein <rk...@rsitex.com>
>>> RSI
>>> 5426 Guadalupe, Suite 100
>>> Austin TX 78751
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Richard Klein (RSI)
>>>> Sent: Tuesday, April 05, 2016 2:32 PM
>>>> To: users@cloudstack.apache.org
>>>> Subject: RE: Cloudstack 4.7 password reset issue.
>>>>
>>>> The snippets for before and after the reboot via console look the
>>>> same
>>> so I
>>>> pasted the 2nd set of message instead of the first. Sorry about that.
>>> I did
>>>> discover that the /var/lib/dhclient/dhclient.leases existed but was
>>> empty. I've
>>>> run across an issue with CentOS 7 where the lease file is missing so
>>>> I
>>> wrote a
>>>> "cloud-dhcp-check" service that makes sure it exists but now I need
>>>> to
>>> validate
>>>> its content. That being said, I have insured that the
>>>> dhclient_leases
>>> was valid
>>>> and replicated the problem.
>>>>
>>>> The cloud-set-guest-xxxx scripts are from the master branch GitHub
>>> repository
>>>> for apaches/cloudstack using the
>>>> "
>>> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-se
>>> t-
>>>> guest-password.in" and the
>>>> "
>>> https://github.com/apache/cloudstack/blob/master/setup/bindir/cloud-se
>>> t-
>>>> guest-sshkey.in" links.
>>>>
>>>> I have attached the entire log from the VR but have some snippets
>>>> below
>>> along
>>>> with the VM client logs and the issue still occurs after fixing the
>>>> dhcp
>>> lease file.
>>>> I did not perform any password resets via the GUI during this process.
>>>
>>>