You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Rohit Yadav <ro...@shapeblue.com> on 2016/03/03 11:43:17 UTC
[DISCUSS] Request for comments: Out-of-band Management for CloudStack
(new feature)
Hi all,
I want to propose this new feature for CloudStack, to support out-of-band management for CloudStack.
In practice, most hosts have a out-of-band management such as iLO or iDRAC (that can support IPMI 2.0 for example). The idea is to allow admins to access this out-of-band management interface. Initially power-state related operations will be supported such as power on/off/reset etc. and a background service to synchronize the power states of the out-of-band management enabled hosts.
Please have a look at the FS and share your feedback on the same:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack
For testing this feature, I’ve created this tool/library that is a fake ipmi server that can be run as a standalone server/tool and also importable to write Marvin tests: https://github.com/bhaisaab/ipmisim
Thanks and regards.
PS. I’ll be demoing this feature in today’s CSEUG meetup in London
[ShapeBlue]<http://www.shapeblue.com>
Rohit Yadav
Software Architect , ShapeBlue
d: | s: +44 203 603 0540<tel:|%20s:%20+44%20203%20603%200540> | m: +91 8826230892<tel:+91%208826230892>
e: rohit.yadav@shapeblue.com | t: <mailto:rohit.yadav@shapeblue.com%20|%20t:> | w: www.shapeblue.com<http://www.shapeblue.com>
a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
[cid:image4b88c6.png@2526f1f5.4a8f6b4e]
Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Hari,
[ShapeBlue]<http://www.shapeblue.com>
Rohit Yadav
Software Architect , ShapeBlue
d: | s: +44 203 603 0540<tel:|%20s:%20+44%20203%20603%200540> | m: +91 8826230892<tel:+91%208826230892>
e: rohit.yadav@shapeblue.com | t: <mailto:rohit.yadav@shapeblue.com%20|%20t:> | w: www.shapeblue.com<http://www.shapeblue.com>
a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
[cid:imagee92091.png@0ce19fb7.479fbee5]
Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
On 07-Mar-2016, at 10:51 AM, Harikrishna Patnala <ha...@accelerite.com>> wrote:
I have a little concern here regarding the baremetal deployment where we use ipmi as power management interface. Are you going to use the existing ipmi util we have or do you have a separate solution for this.
As noted in the FS, the feature is agnostic of a specific out-of-band technology and will by default ship with a ipmitool based driver to support hosts with IPMI 2.0 enabled interface.
The baremetal plugin uses Python based afaik, while the out-of-band ipmitool driver will be a Java wrapper around ‘ipmitool’. The baremetal plugin maintainer in future can refactor their plugin to use the out-of-band management subsystem to manage machines without using a specific/hard-coded driver (ipmi util etc).
Regards.
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Harikrishna Patnala <ha...@accelerite.com>.
Hi Rohit,
Thanks for the proposal and seems like a descent feature to have.
I have a little concern here regarding the baremetal deployment where we use ipmi as power management interface. Are you going to use the existing ipmi util we have or do you have a separate solution for this. If it is the latter case then you may need to consolidate everything to have one solution to avoid redundancy.
Regards,
Harikrishna
On 07-Mar-2016, at 2:51 pm, Rohit Yadav <ro...@shapeblue.com> wrote:
> Hi Ilya,
>
> Thanks for the suggestions.
>
> In the UI, we can easily have a warning statement on the pop up box if host maintenance is enabled. We can also add mechanisms (for example specific APIs or zone/cluster level settings) to allow admins to disable/enable the feature at zone and cluster level without cascading it to the host enabled setting. Will update the FS to reflect these additions.
>
> Regards.
>
>> On 05-Mar-2016, at 7:11 AM, ilya <il...@gmail.com> wrote:
>>
>> Rohit,
>>
>> Great job!
>>
>> Not certain if this was mentioned, but premise behind IPMI integration
>> was partially driven by HA and being able to "fence" the host in
>> question to avoid split brain scenario - as well as other issues when
>> hypervisors malfunction.
>>
>> With that said, Will brings up a good point, what can we offer to avoid
>> accidental power down - while host is functional. I see plenty of
>> curious cloudstack admins who identify a function of a button by
>> pressing on it and saying "lets see what happens". Better yet, some
>> learn on the job on production environments.
>>
>> Do we give the end user a warning of any kind? If possible, i would
>> suggest we give a one line warning - something like "The host is not in
>> Maintenance Mode, proceed at your own risk!" or something to that effect.
>>
>> Obviously this would only be CloudStack UI safeguard and no such warning
>> would be shown when you use APIs directly.
>>
>> Lastly, please consider disable IPMI for Cluster and Zone level, i dont
>> believe we need it for pod. Please dont cascade over every host ipmi
>> object and change their state, instead, have a separate db entry (or
>> else) that tracks this selection for cluster or zone.
>>
>> It could be as simple as configuration setting for zone or cluster.
>>
>> thanks
>> ilya
>>
>>
>> On 3/4/16 1:20 AM, Rohit Yadav wrote:
>>>
>>>
>>> ShapeBlue <http://www.shapeblue.com>
>>> Rohit Yadav
>>> Software Architect , ShapeBlue
>>>
>>> d: * | s: +44 203 603 0540* <tel:|%20s:%20+44%20203%20603%200540>
>>> | m: *+91 8826230892* <tel:+91%208826230892>
>>>
>>> e: *rohit.yadav@shapeblue.com | t: *
>>> <mailto:rohit.yadav@shapeblue.com%20|%20t:> | w:
>>> *www.shapeblue.com* <http://www.shapeblue.com>
>>>
>>> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
>>>
>>> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
>>> Services India LLP is a company incorporated in India and is operated
>>> under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is
>>> a company incorporated in Brasil and is operated under license from
>>> Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The
>>> Republic of South Africa and is traded under license from Shape Blue
>>> Ltd. ShapeBlue is a registered trademark.
>>> This email and any attachments to it may be confidential and are
>>> intended solely for the use of the individual to whom it is addressed.
>>> Any views or opinions expressed are solely those of the author and do
>>> not necessarily represent those of Shape Blue Ltd or related companies.
>>> If you are not the intended recipient of this email, you must neither
>>> take any action based upon its contents, nor copy or show it to anyone.
>>> Please contact the sender if you believe you have received this email in
>>> error.
>>>
>>>
>>>> On 03-Mar-2016, at 12:58 PM, Will Stevens <williamstevens@gmail.com
>>>> <ma...@gmail.com>> wrote:
>>>>
>>>> Maybe I am not understanding something here.
>>>>
>>>> Does this control the power cycle of the management server(s) or the
>>>> hypervisor hosts? The wording is throwing me off.
>>>
>>> Fixed. The feature applies for hypervisor hosts only, not management
>>> server hosts (unless of course, mgmt server host is hypervisor host as
>>> well for example running mgmt server + kvm agnet on a KVM host).
>>>
>>>> I am guessing it is for managing the hypervisor hosts. If this is the
>>>> case,
>>>> does it also handle the "maintenance mode" for the host as well?
>>>
>>> Maintenance mode is a hypervisor semantic, this is not related to the
>>> out-of-band management interface (the BMC, such as iLO, iDRAC) available
>>> on the hypervisor host.
>>>
>>> Even when you enable/disable maintenance mode, you can use a tool like
>>> ‘ipmitool’ to do execute a power management operation such as
>>> on/off/reset etc, so you should be able to perform the same using this
>>> feature. Therefore, presently there is no such enforcement.
>>>
>>>> At least
>>>> with XenServer, if you do a power cycle without putting the host into
>>>> maintenance mode first, all the VRs will have to be restarted on that host
>>>> once it is back up in order for their networking to work again.
>>>
>>> We can put in a rule to avoid executing any power operation when hosts
>>> are put in maintenance mode, though some users may still want to be able
>>> to execute power operations. Comments?
>>>
>>> Regards.
>>>
>>> Find out more about ShapeBlue and our range of CloudStack related services:
>>> IaaS Cloud Design & Build
>>> <http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid
>>> IaaS deployment framework <http://shapeblue.com/csforge/>
>>> CloudStack Consulting <http://shapeblue.com/cloudstack-consultancy/> |
>>> CloudStack Software Engineering
>>> <http://shapeblue.com/cloudstack-software-engineering/>
>>> CloudStack Infrastructure Support
>>> <http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack
>>> Bootcamp Training Courses <http://shapeblue.com/cloudstack-training/>
>
> Regards.
>
> Find out more about ShapeBlue and our range of CloudStack related services:
> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Ilya,
Thanks for the suggestions.
In the UI, we can easily have a warning statement on the pop up box if host maintenance is enabled. We can also add mechanisms (for example specific APIs or zone/cluster level settings) to allow admins to disable/enable the feature at zone and cluster level without cascading it to the host enabled setting. Will update the FS to reflect these additions.
Regards.
> On 05-Mar-2016, at 7:11 AM, ilya <il...@gmail.com> wrote:
>
> Rohit,
>
> Great job!
>
> Not certain if this was mentioned, but premise behind IPMI integration
> was partially driven by HA and being able to "fence" the host in
> question to avoid split brain scenario - as well as other issues when
> hypervisors malfunction.
>
> With that said, Will brings up a good point, what can we offer to avoid
> accidental power down - while host is functional. I see plenty of
> curious cloudstack admins who identify a function of a button by
> pressing on it and saying "lets see what happens". Better yet, some
> learn on the job on production environments.
>
> Do we give the end user a warning of any kind? If possible, i would
> suggest we give a one line warning - something like "The host is not in
> Maintenance Mode, proceed at your own risk!" or something to that effect.
>
> Obviously this would only be CloudStack UI safeguard and no such warning
> would be shown when you use APIs directly.
>
> Lastly, please consider disable IPMI for Cluster and Zone level, i dont
> believe we need it for pod. Please dont cascade over every host ipmi
> object and change their state, instead, have a separate db entry (or
> else) that tracks this selection for cluster or zone.
>
> It could be as simple as configuration setting for zone or cluster.
>
> thanks
> ilya
>
>
> On 3/4/16 1:20 AM, Rohit Yadav wrote:
>>
>>
>> ShapeBlue <http://www.shapeblue.com>
>> Rohit Yadav
>> Software Architect , ShapeBlue
>>
>> d: * | s: +44 203 603 0540* <tel:|%20s:%20+44%20203%20603%200540>
>> | m: *+91 8826230892* <tel:+91%208826230892>
>>
>> e: *rohit.yadav@shapeblue.com | t: *
>> <mailto:rohit.yadav@shapeblue.com%20|%20t:> | w:
>> *www.shapeblue.com* <http://www.shapeblue.com>
>>
>> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
>>
>> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
>> Services India LLP is a company incorporated in India and is operated
>> under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is
>> a company incorporated in Brasil and is operated under license from
>> Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The
>> Republic of South Africa and is traded under license from Shape Blue
>> Ltd. ShapeBlue is a registered trademark.
>> This email and any attachments to it may be confidential and are
>> intended solely for the use of the individual to whom it is addressed.
>> Any views or opinions expressed are solely those of the author and do
>> not necessarily represent those of Shape Blue Ltd or related companies.
>> If you are not the intended recipient of this email, you must neither
>> take any action based upon its contents, nor copy or show it to anyone.
>> Please contact the sender if you believe you have received this email in
>> error.
>>
>>
>>> On 03-Mar-2016, at 12:58 PM, Will Stevens <williamstevens@gmail.com
>>> <ma...@gmail.com>> wrote:
>>>
>>> Maybe I am not understanding something here.
>>>
>>> Does this control the power cycle of the management server(s) or the
>>> hypervisor hosts? The wording is throwing me off.
>>
>> Fixed. The feature applies for hypervisor hosts only, not management
>> server hosts (unless of course, mgmt server host is hypervisor host as
>> well for example running mgmt server + kvm agnet on a KVM host).
>>
>>> I am guessing it is for managing the hypervisor hosts. If this is the
>>> case,
>>> does it also handle the "maintenance mode" for the host as well?
>>
>> Maintenance mode is a hypervisor semantic, this is not related to the
>> out-of-band management interface (the BMC, such as iLO, iDRAC) available
>> on the hypervisor host.
>>
>> Even when you enable/disable maintenance mode, you can use a tool like
>> ‘ipmitool’ to do execute a power management operation such as
>> on/off/reset etc, so you should be able to perform the same using this
>> feature. Therefore, presently there is no such enforcement.
>>
>>> At least
>>> with XenServer, if you do a power cycle without putting the host into
>>> maintenance mode first, all the VRs will have to be restarted on that host
>>> once it is back up in order for their networking to work again.
>>
>> We can put in a rule to avoid executing any power operation when hosts
>> are put in maintenance mode, though some users may still want to be able
>> to execute power operations. Comments?
>>
>> Regards.
>>
>> Find out more about ShapeBlue and our range of CloudStack related services:
>> IaaS Cloud Design & Build
>> <http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid
>> IaaS deployment framework <http://shapeblue.com/csforge/>
>> CloudStack Consulting <http://shapeblue.com/cloudstack-consultancy/> |
>> CloudStack Software Engineering
>> <http://shapeblue.com/cloudstack-software-engineering/>
>> CloudStack Infrastructure Support
>> <http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack
>> Bootcamp Training Courses <http://shapeblue.com/cloudstack-training/>
Regards.
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
Re: [DISCUSS] Request for comments: Out-of-band Management for CloudStack
(new feature)
Posted by ilya <il...@gmail.com>.
Rohit,
Great job!
Not certain if this was mentioned, but premise behind IPMI integration
was partially driven by HA and being able to "fence" the host in
question to avoid split brain scenario - as well as other issues when
hypervisors malfunction.
With that said, Will brings up a good point, what can we offer to avoid
accidental power down - while host is functional. I see plenty of
curious cloudstack admins who identify a function of a button by
pressing on it and saying "lets see what happens". Better yet, some
learn on the job on production environments.
Do we give the end user a warning of any kind? If possible, i would
suggest we give a one line warning - something like "The host is not in
Maintenance Mode, proceed at your own risk!" or something to that effect.
Obviously this would only be CloudStack UI safeguard and no such warning
would be shown when you use APIs directly.
Lastly, please consider disable IPMI for Cluster and Zone level, i dont
believe we need it for pod. Please dont cascade over every host ipmi
object and change their state, instead, have a separate db entry (or
else) that tracks this selection for cluster or zone.
It could be as simple as configuration setting for zone or cluster.
thanks
ilya
On 3/4/16 1:20 AM, Rohit Yadav wrote:
>
>
> ShapeBlue <http://www.shapeblue.com>
> Rohit Yadav
> Software Architect , ShapeBlue
>
> d: * | s: +44 203 603 0540* <tel:|%20s:%20+44%20203%20603%200540>
> | m: *+91 8826230892* <tel:+91%208826230892>
>
> e: *rohit.yadav@shapeblue.com | t: *
> <mailto:rohit.yadav@shapeblue.com%20|%20t:> | w:
> *www.shapeblue.com* <http://www.shapeblue.com>
>
> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
>
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated
> under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is
> a company incorporated in Brasil and is operated under license from
> Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The
> Republic of South Africa and is traded under license from Shape Blue
> Ltd. ShapeBlue is a registered trademark.
> This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed.
> Any views or opinions expressed are solely those of the author and do
> not necessarily represent those of Shape Blue Ltd or related companies.
> If you are not the intended recipient of this email, you must neither
> take any action based upon its contents, nor copy or show it to anyone.
> Please contact the sender if you believe you have received this email in
> error.
>
>
>> On 03-Mar-2016, at 12:58 PM, Will Stevens <williamstevens@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> Maybe I am not understanding something here.
>>
>> Does this control the power cycle of the management server(s) or the
>> hypervisor hosts? The wording is throwing me off.
>
> Fixed. The feature applies for hypervisor hosts only, not management
> server hosts (unless of course, mgmt server host is hypervisor host as
> well for example running mgmt server + kvm agnet on a KVM host).
>
>> I am guessing it is for managing the hypervisor hosts. If this is the
>> case,
>> does it also handle the "maintenance mode" for the host as well?
>
> Maintenance mode is a hypervisor semantic, this is not related to the
> out-of-band management interface (the BMC, such as iLO, iDRAC) available
> on the hypervisor host.
>
> Even when you enable/disable maintenance mode, you can use a tool like
> ‘ipmitool’ to do execute a power management operation such as
> on/off/reset etc, so you should be able to perform the same using this
> feature. Therefore, presently there is no such enforcement.
>
>> At least
>> with XenServer, if you do a power cycle without putting the host into
>> maintenance mode first, all the VRs will have to be restarted on that host
>> once it is back up in order for their networking to work again.
>
> We can put in a rule to avoid executing any power operation when hosts
> are put in maintenance mode, though some users may still want to be able
> to execute power operations. Comments?
>
> Regards.
>
> Find out more about ShapeBlue and our range of CloudStack related services:
> IaaS Cloud Design & Build
> <http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid
> IaaS deployment framework <http://shapeblue.com/csforge/>
> CloudStack Consulting <http://shapeblue.com/cloudstack-consultancy/> |
> CloudStack Software Engineering
> <http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support
> <http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack
> Bootcamp Training Courses <http://shapeblue.com/cloudstack-training/>
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Rohit Yadav <ro...@shapeblue.com>.
[ShapeBlue]<http://www.shapeblue.com>
Rohit Yadav
Software Architect , ShapeBlue
d: | s: +44 203 603 0540<tel:|%20s:%20+44%20203%20603%200540> | m: +91 8826230892<tel:+91%208826230892>
e: rohit.yadav@shapeblue.com | t: <mailto:rohit.yadav@shapeblue.com%20|%20t:> | w: www.shapeblue.com<http://www.shapeblue.com>
a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
[cid:image1c101a.png@324a4be9.4c92bbb1]
Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
On 03-Mar-2016, at 12:58 PM, Will Stevens <wi...@gmail.com>> wrote:
Maybe I am not understanding something here.
Does this control the power cycle of the management server(s) or the
hypervisor hosts? The wording is throwing me off.
Fixed. The feature applies for hypervisor hosts only, not management server hosts (unless of course, mgmt server host is hypervisor host as well for example running mgmt server + kvm agnet on a KVM host).
I am guessing it is for managing the hypervisor hosts. If this is the case,
does it also handle the "maintenance mode" for the host as well?
Maintenance mode is a hypervisor semantic, this is not related to the out-of-band management interface (the BMC, such as iLO, iDRAC) available on the hypervisor host.
Even when you enable/disable maintenance mode, you can use a tool like ‘ipmitool’ to do execute a power management operation such as on/off/reset etc, so you should be able to perform the same using this feature. Therefore, presently there is no such enforcement.
At least
with XenServer, if you do a power cycle without putting the host into
maintenance mode first, all the VRs will have to be restarted on that host
once it is back up in order for their networking to work again.
We can put in a rule to avoid executing any power operation when hosts are put in maintenance mode, though some users may still want to be able to execute power operations. Comments?
Regards.
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Will Stevens <wi...@gmail.com>.
Maybe I am not understanding something here.
Does this control the power cycle of the management server(s) or the
hypervisor hosts? The wording is throwing me off.
I am guessing it is for managing the hypervisor hosts. If this is the case,
does it also handle the "maintenance mode" for the host as well? At least
with XenServer, if you do a power cycle without putting the host into
maintenance mode first, all the VRs will have to be restarted on that host
once it is back up in order for their networking to work again.
How does this fit into this?
On Mar 3, 2016 5:43 AM, "Rohit Yadav" <ro...@shapeblue.com> wrote:
> Hi all,
>
> I want to propose this new feature for CloudStack, to support out-of-band
> management for CloudStack.
>
> In practice, most hosts have a out-of-band management such as iLO or iDRAC
> (that can support IPMI 2.0 for example). The idea is to allow admins to
> access this out-of-band management interface. Initially power-state related
> operations will be supported such as power on/off/reset etc. and a
> background service to synchronize the power states of the out-of-band
> management enabled hosts.
>
> Please have a look at the FS and share your feedback on the same:
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack
>
> For testing this feature, I’ve created this tool/library that is a fake
> ipmi server that can be run as a standalone server/tool and also importable
> to write Marvin tests: https://github.com/bhaisaab/ipmisim
>
> Thanks and regards.
> PS. I’ll be demoing this feature in today’s CSEUG meetup in London
>
> [image: ShapeBlue] <http://www.shapeblue.com>
> Rohit Yadav
> Software Architect , ShapeBlue
> d: * | s: +44 203 603 0540* <%7C%20s:%20+44%20203%20603%200540> | m:
> *+91 8826230892* <+91%208826230892>
> e: *rohit.yadav@shapeblue.com | t: *
> <rohit.yadav@shapeblue.com%20%7C%20t:> | w: *www.shapeblue.com*
> <http://www.shapeblue.com>
> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated under
> license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a
> company incorporated in Brasil and is operated under license from Shape
> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of
> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is
> a registered trademark.
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error.
>
>
> Find out more about ShapeBlue and our range of CloudStack related services:
> IaaS Cloud Design & Build
> <http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid
> IaaS deployment framework <http://shapeblue.com/csforge/>
> CloudStack Consulting <http://shapeblue.com/cloudstack-consultancy/> | CloudStack
> Software Engineering
> <http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support
> <http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack
> Bootcamp Training Courses <http://shapeblue.com/cloudstack-training/>
>
RE: [DISCUSS] Request for comments: Out-of-band Management
for CloudStack (new feature)
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi all,
The feature is ready for code review now: https://github.com/apache/cloudstack/pull/1502
Thanks.
Regards,
Rohit Yadav
rohit.yadav@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
Sent: Thursday, March 3, 2016 4:13 PM
To: dev@cloudstack.apache.org
Subject: [DISCUSS] Request for comments: Out-of-band Management for CloudStack (new feature)
Hi all,
I want to propose this new feature for CloudStack, to support out-of-band management for CloudStack.
In practice, most hosts have a out-of-band management such as iLO or iDRAC (that can support IPMI 2.0 for example). The idea is to allow admins to access this out-of-band management interface. Initially power-state related operations will be supported such as power on/off/reset etc. and a background service to synchronize the power states of the out-of-band management enabled hosts.
Please have a look at the FS and share your feedback on the same:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack
For testing this feature, I’ve created this tool/library that is a fake ipmi server that can be run as a standalone server/tool and also importable to write Marvin tests: https://github.com/bhaisaab/ipmisim
Thanks and regards.
PS. I’ll be demoing this feature in today’s CSEUG meetup in London
[ShapeBlue]<http://www.shapeblue.com>
Rohit Yadav
Software Architect
,
ShapeBlue
d:
| s: +44 203 603 0540<tel:|%20s:%20+44%20203%20603%200540>
|
m:
+91 8826230892<tel:+91%208826230892>
e:
rohit.yadav@shapeblue.com | t: <mailto:rohit.yadav@shapeblue.com%20|%20t:>
|
w:
www.shapeblue.com<http://www.shapeblue.com>
a:
53 Chandos Place, Covent Garden London WC2N 4HS UK
[cid:image4b88c6.png@2526f1f5.4a8f6b4e]
Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build/> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Ahmad,
Thanks. Good to see you again on the ML, after a long break.
The IPMI simulation is not tied with the ACS simulator (plugin) but it’s a separate Python based tool (also an importable library). Since it’s Python based, it would be difficult to tied it with the simulator plugin. Instead of using ipmisim or the default ipmitool driver that ships with this feature, the simulator plugin can have a separate dummy driver implementation.
The Marvin test will be Python based, so it will be easiest to tie ipmisim with it (no external setup needed for example); while to test this feature manually one can easily install ipmisim and run it separately.
[ShapeBlue]<http://www.shapeblue.com>
Rohit Yadav
Software Architect , ShapeBlue
d: | s: +44 203 603 0540<tel:|%20s:%20+44%20203%20603%200540> | m: +91 8826230892<tel:+91%208826230892>
e: rohit.yadav@shapeblue.com | t: <mailto:rohit.yadav@shapeblue.com%20|%20t:> | w: www.shapeblue.com<http://www.shapeblue.com>
a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
[cid:image783119.png@1a865698.4eafe4a5]
Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
On 05-Mar-2016, at 7:54 AM, Ahmad Emneina <ae...@gmail.com>> wrote:
qq: is the ipmi simulation tied, or planned
to be tied to 'the simulator'? It would be interesting to see those tied
together. regardless... awesome work.
Regards.
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Ahmad Emneina <ae...@gmail.com>.
+1, this looks like a solid feature Rohit. I'd love to see where this goes
in the future... Maybe automatic power down of hosts idling, and powering
on hosts based on load demands. qq: is the ipmi simulation tied, or planned
to be tied to 'the simulator'? It would be interesting to see those tied
together. regardless... awesome work.
On Thu, Mar 3, 2016 at 2:43 AM, Rohit Yadav <ro...@shapeblue.com>
wrote:
> Hi all,
>
> I want to propose this new feature for CloudStack, to support out-of-band
> management for CloudStack.
>
> In practice, most hosts have a out-of-band management such as iLO or iDRAC
> (that can support IPMI 2.0 for example). The idea is to allow admins to
> access this out-of-band management interface. Initially power-state related
> operations will be supported such as power on/off/reset etc. and a
> background service to synchronize the power states of the out-of-band
> management enabled hosts.
>
> Please have a look at the FS and share your feedback on the same:
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack
>
> For testing this feature, I’ve created this tool/library that is a fake
> ipmi server that can be run as a standalone server/tool and also importable
> to write Marvin tests: https://github.com/bhaisaab/ipmisim
>
> Thanks and regards.
> PS. I’ll be demoing this feature in today’s CSEUG meetup in London
>
> [image: ShapeBlue] <http://www.shapeblue.com>
> Rohit Yadav
> Software Architect , ShapeBlue
> d: * | s: +44 203 603 0540* <%7C%20s:%20+44%20203%20603%200540> | m:
> *+91 8826230892* <+91%208826230892>
> e: *rohit.yadav@shapeblue.com | t: *
> <rohit.yadav@shapeblue.com%20%7C%20t:> | w: *www.shapeblue.com*
> <http://www.shapeblue.com>
> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated under
> license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a
> company incorporated in Brasil and is operated under license from Shape
> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of
> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is
> a registered trademark.
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error.
>
>
> Find out more about ShapeBlue and our range of CloudStack related services:
> IaaS Cloud Design & Build
> <http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid
> IaaS deployment framework <http://shapeblue.com/csforge/>
> CloudStack Consulting <http://shapeblue.com/cloudstack-consultancy/> | CloudStack
> Software Engineering
> <http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support
> <http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack
> Bootcamp Training Courses <http://shapeblue.com/cloudstack-training/>
>
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Erik Weber <te...@gmail.com>.
Sounds like a decent addition to me, +1
--
Erik
On Thu, Mar 3, 2016 at 11:43 AM, Rohit Yadav <ro...@shapeblue.com>
wrote:
> Hi all,
>
> I want to propose this new feature for CloudStack, to support out-of-band
> management for CloudStack.
>
> In practice, most hosts have a out-of-band management such as iLO or iDRAC
> (that can support IPMI 2.0 for example). The idea is to allow admins to
> access this out-of-band management interface. Initially power-state related
> operations will be supported such as power on/off/reset etc. and a
> background service to synchronize the power states of the out-of-band
> management enabled hosts.
>
> Please have a look at the FS and share your feedback on the same:
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack
>
> For testing this feature, I’ve created this tool/library that is a fake
> ipmi server that can be run as a standalone server/tool and also importable
> to write Marvin tests: https://github.com/bhaisaab/ipmisim
>
> Thanks and regards.
> PS. I’ll be demoing this feature in today’s CSEUG meetup in London
>
> [image: ShapeBlue] <http://www.shapeblue.com>
> Rohit Yadav
> Software Architect , ShapeBlue
> d: * | s: +44 203 603 0540* <%7C%20s:%20+44%20203%20603%200540> | m:
> *+91 8826230892* <+91%208826230892>
> e: *rohit.yadav@shapeblue.com | t: *
> <rohit.yadav@shapeblue.com%20%7C%20t:> | w: *www.shapeblue.com*
> <http://www.shapeblue.com>
> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated under
> license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a
> company incorporated in Brasil and is operated under license from Shape
> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of
> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is
> a registered trademark.
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error.
>
>
> Find out more about ShapeBlue and our range of CloudStack related services:
> IaaS Cloud Design & Build
> <http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid
> IaaS deployment framework <http://shapeblue.com/csforge/>
> CloudStack Consulting <http://shapeblue.com/cloudstack-consultancy/> | CloudStack
> Software Engineering
> <http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support
> <http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack
> Bootcamp Training Courses <http://shapeblue.com/cloudstack-training/>
>
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Koushik Das <ko...@accelerite.com>.
Thanks Rohit for the clarifications. I will get back if there are additional questions.
-Koushik
________________________________
From: Rohit Yadav <ro...@shapeblue.com>
Sent: Friday, March 4, 2016 3:29 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Request for comments: Out-of-band Management for CloudStack (new feature)
Hi Koushik,
[ShapeBlue]<http://www.shapeblue.com>
Rohit Yadav
Software Architect , ShapeBlue
d: | s: +44 203 603 0540<tel:|%20s:%20+44%20203%20603%200540> | m: +91 8826230892<tel:+91%208826230892>
e: rohit.yadav@shapeblue.com | t: <mailto:rohit.yadav@shapeblue.com%20|%20t:> | w: www.shapeblue.com<http://www.shapeblue.com>
a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
[cid:image6d46e0.png@6cbf0119.47b60f2c]
Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
On 03-Mar-2016, at 2:38 PM, Koushik Das <ko...@accelerite.com>> wrote:
Host maintenance is already there to address the use cases mentioned. What additional value would this feature add?
Host maintenance and out-of-band power management are both different things, and don’t address the same use-cases.
Also if it is just providing the IPMI based operations then it is only limited to the hosts already added to CS, what about new hosts not added in CS? What if host IP/name/password etc. which are already stored in CS needs to be changed?
Limited to hypervisor hosts that have been already added in CloudStack.
Nothing we can do about (hypervisor) hosts, CloudStack does not know of.
The out-of-band management interface (such as iLo, iDRAC based) is a separate computer (BMC, https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface#Baseboard_management_controller) with a separate NIC, power etc. it’s not same as the (hypervisor) host itself. Therefore, has its own IP address etc. If the out-of-band management credentials (ip, port, username, password) is changed then this information needs to be updated in CloudStack as well.
If an admin wants to change the password of user for the out-of-band management interface for a (hypervisor) host, they can use the change oobm password API which will change the password on the BMC and also update this information in CloudStack.
To understand out-of-band management related concepts, you can play with the ‘ipmisim’ tool that I created for this testing this feature:
1. pip install —upgrade ipmisim
2. Run ipmisim in a separate terminal
3. Run ipmitool to see the power status, issue some power commans: (osx users, do a brew install ipmitool; don’t use apple/bsd ipmitool)
ipmitool -I lanplus -H localhost -p 9001 -R1 -U admin -P password chassis power status -v # by default server should report Off
ipmitool -I lanplus -H localhost -p 9001 -R1 -U admin -P password chassis power on -v
ipmitool -I lanplus -H localhost -p 9001 -R1 -U admin -P password chassis power status -v # see the server would report server is On now
Some questions on use cases.
* Restarting stalled/failed hosts
[Koushik] How do you determine if a host stalled/failed? What happens if the host lost network connectivity intermittently?
It depends, if you’re asking if this is done automatically — no, it’s a manual operation (for now, though could be useful if we want to do host HA and fencing in future).
The admin can assess their situation, for example check the power state with this feature (which they normally would do by running ipmitool chassis power status command) and if the power state is On they can turn them On by issuing a power action (from UI/API).
* Powering off under-utilised hosts
[Koushik] What is the definition of "under-utilised" - no VMs or based on some resource threshold?
It depends, the main idea is that you can execute a power operation.
By 'Under-utilised hosts’ I mean to say hosts without any VMs on them.
If there are some running VMs (HA and non-HA) what happens to them?
Don’t understand what you’re trying to ask.
I suppose bad things will happen to non-HA VMs if you turn off (hypervisor) hosts.
* Powering on hosts for provisioning and to increase capacity (for hosts already provisioned)
[Koushik] Again how do you power on new hosts not yet added to CS?
For hosts already provisioned or added to CloudStack.
* Allowing system administrators to see the current power state of the host
[Koushik] When would the power state differ from the existing host state shown in CS? There can be certain corner scenarios when CS may not determine the host state conclusively (e.g. last host in cluster), but otherwise shouldn't it be the same?
Power state can differ when you’re doing a power operation (such as on/off/reset etc) or say in case of a power outage.
See the FSM for details. CloudStack won’t be able to determine the out-of-band management power state of a hypervisor host in case of (1) network connectivity issue when trying to connect to the BMC (the power state will be put in Unknown state, events issued), (2) the password/credentials are incorrect (alert will be issued, admins will get an email if enabled).
Regards.
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Koushik,
[ShapeBlue]<http://www.shapeblue.com>
Rohit Yadav
Software Architect , ShapeBlue
d: | s: +44 203 603 0540<tel:|%20s:%20+44%20203%20603%200540> | m: +91 8826230892<tel:+91%208826230892>
e: rohit.yadav@shapeblue.com | t: <mailto:rohit.yadav@shapeblue.com%20|%20t:> | w: www.shapeblue.com<http://www.shapeblue.com>
a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
[cid:image6d46e0.png@6cbf0119.47b60f2c]
Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
On 03-Mar-2016, at 2:38 PM, Koushik Das <ko...@accelerite.com>> wrote:
Host maintenance is already there to address the use cases mentioned. What additional value would this feature add?
Host maintenance and out-of-band power management are both different things, and don’t address the same use-cases.
Also if it is just providing the IPMI based operations then it is only limited to the hosts already added to CS, what about new hosts not added in CS? What if host IP/name/password etc. which are already stored in CS needs to be changed?
Limited to hypervisor hosts that have been already added in CloudStack.
Nothing we can do about (hypervisor) hosts, CloudStack does not know of.
The out-of-band management interface (such as iLo, iDRAC based) is a separate computer (BMC, https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface#Baseboard_management_controller) with a separate NIC, power etc. it’s not same as the (hypervisor) host itself. Therefore, has its own IP address etc. If the out-of-band management credentials (ip, port, username, password) is changed then this information needs to be updated in CloudStack as well.
If an admin wants to change the password of user for the out-of-band management interface for a (hypervisor) host, they can use the change oobm password API which will change the password on the BMC and also update this information in CloudStack.
To understand out-of-band management related concepts, you can play with the ‘ipmisim’ tool that I created for this testing this feature:
1. pip install —upgrade ipmisim
2. Run ipmisim in a separate terminal
3. Run ipmitool to see the power status, issue some power commans: (osx users, do a brew install ipmitool; don’t use apple/bsd ipmitool)
ipmitool -I lanplus -H localhost -p 9001 -R1 -U admin -P password chassis power status -v # by default server should report Off
ipmitool -I lanplus -H localhost -p 9001 -R1 -U admin -P password chassis power on -v
ipmitool -I lanplus -H localhost -p 9001 -R1 -U admin -P password chassis power status -v # see the server would report server is On now
Some questions on use cases.
* Restarting stalled/failed hosts
[Koushik] How do you determine if a host stalled/failed? What happens if the host lost network connectivity intermittently?
It depends, if you’re asking if this is done automatically — no, it’s a manual operation (for now, though could be useful if we want to do host HA and fencing in future).
The admin can assess their situation, for example check the power state with this feature (which they normally would do by running ipmitool chassis power status command) and if the power state is On they can turn them On by issuing a power action (from UI/API).
* Powering off under-utilised hosts
[Koushik] What is the definition of "under-utilised" - no VMs or based on some resource threshold?
It depends, the main idea is that you can execute a power operation.
By 'Under-utilised hosts’ I mean to say hosts without any VMs on them.
If there are some running VMs (HA and non-HA) what happens to them?
Don’t understand what you’re trying to ask.
I suppose bad things will happen to non-HA VMs if you turn off (hypervisor) hosts.
* Powering on hosts for provisioning and to increase capacity (for hosts already provisioned)
[Koushik] Again how do you power on new hosts not yet added to CS?
For hosts already provisioned or added to CloudStack.
* Allowing system administrators to see the current power state of the host
[Koushik] When would the power state differ from the existing host state shown in CS? There can be certain corner scenarios when CS may not determine the host state conclusively (e.g. last host in cluster), but otherwise shouldn't it be the same?
Power state can differ when you’re doing a power operation (such as on/off/reset etc) or say in case of a power outage.
See the FSM for details. CloudStack won’t be able to determine the out-of-band management power state of a hypervisor host in case of (1) network connectivity issue when trying to connect to the BMC (the power state will be put in Unknown state, events issued), (2) the password/credentials are incorrect (alert will be issued, admins will get an email if enabled).
Regards.
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Koushik Das <ko...@accelerite.com>.
Hi Rohit,
Host maintenance is already there to address the use cases mentioned. What additional value would this feature add? Also if it is just providing the IPMI based operations then it is only limited to the hosts already added to CS, what about new hosts not added in CS? What if host IP/name/password etc. which are already stored in CS needs to be changed?
Some questions on use cases.
* Restarting stalled/failed hosts
[Koushik] How do you determine if a host stalled/failed? What happens if the host lost network connectivity intermittently?
* Powering off under-utilised hosts
[Koushik] What is the definition of "under-utilised" - no VMs or based on some resource threshold? If there are some running VMs (HA and non-HA) what happens to them?
* Powering on hosts for provisioning and to increase capacity (for hosts already provisioned)
[Koushik] Again how do you power on new hosts not yet added to CS?
* Allowing system administrators to see the current power state of the host
[Koushik] When would the power state differ from the existing host state shown in CS? There can be certain corner scenarios when CS may not determine the host state conclusively (e.g. last host in cluster), but otherwise shouldn't it be the same?
Thanks,
Koushik
________________________________
From: Rohit Yadav <ro...@shapeblue.com>
Sent: Thursday, March 3, 2016 4:13 PM
To: dev@cloudstack.apache.org
Subject: [DISCUSS] Request for comments: Out-of-band Management for CloudStack (new feature)
Hi all,
I want to propose this new feature for CloudStack, to support out-of-band management for CloudStack.
In practice, most hosts have a out-of-band management such as iLO or iDRAC (that can support IPMI 2.0 for example). The idea is to allow admins to access this out-of-band management interface. Initially power-state related operations will be supported such as power on/off/reset etc. and a background service to synchronize the power states of the out-of-band management enabled hosts.
Please have a look at the FS and share your feedback on the same:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack
For testing this feature, I've created this tool/library that is a fake ipmi server that can be run as a standalone server/tool and also importable to write Marvin tests: https://github.com/bhaisaab/ipmisim
Thanks and regards.
PS. I'll be demoing this feature in today's CSEUG meetup in London
[ShapeBlue]<http://www.shapeblue.com>
Rohit Yadav
Software Architect , ShapeBlue
d: | s: +44 203 603 0540<tel:|%20s:%20+44%20203%20603%200540> | m: +91 8826230892<tel:+91%208826230892>
e: rohit.yadav@shapeblue.com | t: <mailto:rohit.yadav@shapeblue.com%20|%20t:> | w: www.shapeblue.com<http://www.shapeblue.com>
a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
[cid:image4b88c6.png@2526f1f5.4a8f6b4e]
Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge - rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: [DISCUSS] Request for comments: Out-of-band Management for
CloudStack (new feature)
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Wido,
The feature does not assumption IPv4 or IPv6, it’s upto the driver and the address you provide. Presently, the driver simply forks out a ipmitool process with a bunch of validated arguments. So, if the driver can reach to the specified address/port (address can be a fqdn, ipv4 or ipv6 address) it will work with your ipv6 servers. (if you can do a ipmitool <args> <cmd> from the mgmt server host, the feature will work in your env)
> On 03-Mar-2016, at 11:13 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
> Short note (on tablet), please make sure it supports IPv6 :)
>
> We at PCextreme manage our IPMI for example over IPv6 only.
>
> Wido
>
>> Op 3 mrt. 2016 om 11:43 heeft Rohit Yadav <ro...@shapeblue.com> het volgende geschreven:
>>
>> Hi all,
>>
>> I want to propose this new feature for CloudStack, to support out-of-band management for CloudStack.
>>
>> In practice, most hosts have a out-of-band management such as iLO or iDRAC (that can support IPMI 2.0 for example). The idea is to allow admins to access this out-of-band management interface. Initially power-state related operations will be supported such as power on/off/reset etc. and a background service to synchronize the power states of the out-of-band management enabled hosts.
>>
>> Please have a look at the FS and share your feedback on the same:
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack
>>
>> For testing this feature, I’ve created this tool/library that is a fake ipmi server that can be run as a standalone server/tool and also importable to write Marvin tests: https://github.com/bhaisaab/ipmisim
>>
>> Thanks and regards.
>> PS. I’ll be demoing this feature in today’s CSEUG meetup in London
>>
>>
>> Rohit Yadav
>> Software Architect , ShapeBlue
>> d: | s: +44 203 603 0540 | m: +91 8826230892
>> e: rohit.yadav@shapeblue.com | t: | w: www.shapeblue.com
>> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
>>
>> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>> This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
>>
>>
>> Find out more about ShapeBlue and our range of CloudStack related services:
>> IaaS Cloud Design & Build | CSForge – rapid IaaS deployment framework
>> CloudStack Consulting | CloudStack Software Engineering
>> CloudStack Infrastructure Support | CloudStack Bootcamp Training Courses
Regards.
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
Re: [DISCUSS] Request for comments: Out-of-band Management for CloudStack (new feature)
Posted by Wido den Hollander <wi...@widodh.nl>.
Short note (on tablet), please make sure it supports IPv6 :)
We at PCextreme manage our IPMI for example over IPv6 only.
Wido
> Op 3 mrt. 2016 om 11:43 heeft Rohit Yadav <ro...@shapeblue.com> het volgende geschreven:
>
> Hi all,
>
> I want to propose this new feature for CloudStack, to support out-of-band management for CloudStack.
>
> In practice, most hosts have a out-of-band management such as iLO or iDRAC (that can support IPMI 2.0 for example). The idea is to allow admins to access this out-of-band management interface. Initially power-state related operations will be supported such as power on/off/reset etc. and a background service to synchronize the power states of the out-of-band management enabled hosts.
>
> Please have a look at the FS and share your feedback on the same:
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack
>
> For testing this feature, I’ve created this tool/library that is a fake ipmi server that can be run as a standalone server/tool and also importable to write Marvin tests: https://github.com/bhaisaab/ipmisim
>
> Thanks and regards.
> PS. I’ll be demoing this feature in today’s CSEUG meetup in London
>
>
> Rohit Yadav
> Software Architect , ShapeBlue
> d: | s: +44 203 603 0540 | m: +91 8826230892
> e: rohit.yadav@shapeblue.com | t: | w: www.shapeblue.com
> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
>
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
> This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.
>
>
> Find out more about ShapeBlue and our range of CloudStack related services:
> IaaS Cloud Design & Build | CSForge – rapid IaaS deployment framework
> CloudStack Consulting | CloudStack Software Engineering
> CloudStack Infrastructure Support | CloudStack Bootcamp Training Courses