You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by "Jarosław Dzbęk (JIRA)" <ji...@apache.org> on 2012/09/20 11:11:07 UTC

[jira] [Created] (CLOUDSTACK-155) Storage HA checks cause Compute Node reboot when storage in Maintenence Mode

Jarosław Dzbęk created CLOUDSTACK-155:
-----------------------------------------

             Summary: Storage HA checks cause Compute Node reboot when storage in Maintenence Mode
                 Key: CLOUDSTACK-155
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-155
             Project: CloudStack
          Issue Type: Bug
          Components: KVM, Volumes
    Affects Versions: pre-4.0.0
            Reporter: Jarosław Dzbęk



We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS

When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.

The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CLOUDSTACK-155) Storage HA checks cause Compute Node reboot when storage in Maintenence Mode

Posted by "Jarosław Dzbęk (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CLOUDSTACK-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jarosław Dzbęk updated CLOUDSTACK-155:
--------------------------------------

    Description: 
We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS

When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.

The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.

  was:

We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS

When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.

The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.


    
> Storage HA checks cause Compute Node reboot when storage in Maintenence Mode
> ----------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-155
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-155
>             Project: CloudStack
>          Issue Type: Bug
>          Components: KVM, Volumes
>    Affects Versions: pre-4.0.0
>            Reporter: Jarosław Dzbęk
>         Attachments: ha.log
>
>
> We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS
> When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.
> The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CLOUDSTACK-155) HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode

Posted by "Sudha Ponnaganti (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CLOUDSTACK-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sudha Ponnaganti updated CLOUDSTACK-155:
----------------------------------------

    Fix Version/s: pre-4.0.0
         Assignee: Anthony Xu
    
> HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode
> ---------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-155
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-155
>             Project: CloudStack
>          Issue Type: Bug
>          Components: KVM, Volumes
>    Affects Versions: pre-4.0.0
>            Reporter: Jarosław Dzbęk
>            Assignee: Anthony Xu
>             Fix For: pre-4.0.0
>
>         Attachments: ha.log
>
>
> We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS
> When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.
> The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CLOUDSTACK-155) HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode

Posted by "Alex Huang (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CLOUDSTACK-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459979#comment-13459979 ] 

Alex Huang commented on CLOUDSTACK-155:
---------------------------------------

A workaround for this problem is to disable the heartbeat process when doing storage maintenance manually. 
                
> HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode
> ---------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-155
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-155
>             Project: CloudStack
>          Issue Type: Bug
>          Components: KVM, Volumes
>    Affects Versions: pre-4.0.0
>            Reporter: Jarosław Dzbęk
>            Assignee: edison su
>             Fix For: 4.1.0
>
>         Attachments: ha.log
>
>
> We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS
> When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.
> The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CLOUDSTACK-155) HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode

Posted by "Anthony Xu (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CLOUDSTACK-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459807#comment-13459807 ] 

Anthony Xu commented on CLOUDSTACK-155:
---------------------------------------

Please assign it to whoever is working on KVM
                
> HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode
> ---------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-155
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-155
>             Project: CloudStack
>          Issue Type: Bug
>          Components: KVM, Volumes
>    Affects Versions: pre-4.0.0
>            Reporter: Jarosław Dzbęk
>            Assignee: Anthony Xu
>             Fix For: pre-4.0.0
>
>         Attachments: ha.log
>
>
> We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS
> When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.
> The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CLOUDSTACK-155) Storage HA checks cause Compute Node reboot when storage in Maintenence Mode

Posted by "Jarosław Dzbęk (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CLOUDSTACK-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jarosław Dzbęk updated CLOUDSTACK-155:
--------------------------------------

    Attachment: ha.log
    
> Storage HA checks cause Compute Node reboot when storage in Maintenence Mode
> ----------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-155
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-155
>             Project: CloudStack
>          Issue Type: Bug
>          Components: KVM, Volumes
>    Affects Versions: pre-4.0.0
>            Reporter: Jarosław Dzbęk
>         Attachments: ha.log
>
>
> We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS
> When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.
> The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CLOUDSTACK-155) HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode

Posted by "Alex Huang (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CLOUDSTACK-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alex Huang updated CLOUDSTACK-155:
----------------------------------

    Fix Version/s:     (was: pre-4.0.0)
                   4.1.0
    
> HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode
> ---------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-155
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-155
>             Project: CloudStack
>          Issue Type: Bug
>          Components: KVM, Volumes
>    Affects Versions: pre-4.0.0
>            Reporter: Jarosław Dzbęk
>            Assignee: edison su
>             Fix For: 4.1.0
>
>         Attachments: ha.log
>
>
> We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS
> When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.
> The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CLOUDSTACK-155) HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode

Posted by "Jarosław Dzbęk (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CLOUDSTACK-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jarosław Dzbęk updated CLOUDSTACK-155:
--------------------------------------

    Summary: HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode  (was: Storage HA checks cause Compute Node reboot when storage in Maintenence Mode)
    
> HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode
> ---------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-155
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-155
>             Project: CloudStack
>          Issue Type: Bug
>          Components: KVM, Volumes
>    Affects Versions: pre-4.0.0
>            Reporter: Jarosław Dzbęk
>         Attachments: ha.log
>
>
> We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS
> When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.
> The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CLOUDSTACK-155) HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode

Posted by "edison su (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CLOUDSTACK-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

edison su reassigned CLOUDSTACK-155:
------------------------------------

    Assignee: edison su  (was: Anthony Xu)
    
> HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode
> ---------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-155
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-155
>             Project: CloudStack
>          Issue Type: Bug
>          Components: KVM, Volumes
>    Affects Versions: pre-4.0.0
>            Reporter: Jarosław Dzbęk
>            Assignee: edison su
>             Fix For: pre-4.0.0
>
>         Attachments: ha.log
>
>
> We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS
> When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.
> The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CLOUDSTACK-155) HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode

Posted by "Alex Huang (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CLOUDSTACK-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459976#comment-13459976 ] 

Alex Huang commented on CLOUDSTACK-155:
---------------------------------------

Talked to Edison.  This is a existing problem so I'm moving it to 4.1.  4.0 currently is closed for new bugs.
                
> HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode
> ---------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-155
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-155
>             Project: CloudStack
>          Issue Type: Bug
>          Components: KVM, Volumes
>    Affects Versions: pre-4.0.0
>            Reporter: Jarosław Dzbęk
>            Assignee: edison su
>             Fix For: 4.1.0
>
>         Attachments: ha.log
>
>
> We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS
> When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.
> The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CLOUDSTACK-155) HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode

Posted by "Anthony Xu (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CLOUDSTACK-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460019#comment-13460019 ] 

Anthony Xu commented on CLOUDSTACK-155:
---------------------------------------

XenServer doesn't have this issue,
In XenServer, there is file to record which SRs need to check, when PS is put into maintenance mode, then the SR is removed from this file, hearbeat process will not check hearbeat on this PS.
                
> HA checks lead to unnecessary Compute Node reboot when Primary Storage is in Maintenance Mode
> ---------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-155
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-155
>             Project: CloudStack
>          Issue Type: Bug
>          Components: KVM, Volumes
>    Affects Versions: pre-4.0.0
>            Reporter: Jarosław Dzbęk
>            Assignee: edison su
>             Fix For: 4.1.0
>
>         Attachments: ha.log
>
>
> We are using Cloudstack 3.0.2 using KVM on Ubuntu 12.04 with PrimaryStorage over NFS
> When we enable Maintenance Mode on a PrimaryStorage and disconnect the storage machine for reconfiguration the heartbeat checks on the KVM lead to an unwanted reboot of KVM Compute Nodes.
> The /usr/lib64/cloud/agent/scripts/vm/hypervisor/kvm/kvmheartbeat.sh script is correctly identifying a persistent problem with the disconnected storage pool. The problem is that this pool was intentionally put into maintenance mode and MAY be unavailable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira