You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Jayanth Reddy <ja...@gmail.com> on 2022/05/02 04:34:34 UTC

Database High Availability

Hello guys,

    How are you doing database High Availability? Any inputs on DB
Clustering and CloudStack configuration would really help me.

Re: Database High Availability

Posted by Jayanth Reddy <ja...@gmail.com>.
Sure, thanks a lot!

On Tue, May 3, 2022 at 12:53 PM Ivan Kudryavtsev <iv...@bw-sw.com> wrote:

> Well,
>
> You would better consult with real-life mysql experts, as for me, I
> referred to great severalnines.com articles like:
>
> https://severalnines.com/resources/database-management-tutorials/galera-cluster-mysql-tutorial
>
> https://severalnines.com/database-blog/avoiding-deadlocks-galera-setting-haproxy-single-node-writes-and-multi-node-reads
>
> Just take a look for best practices there.
>
>
>
> On Tue, May 3, 2022 at 10:18 AM Jayanth Reddy <ja...@gmail.com>
> wrote:
>
> > Hi,
> >
> >  Thanks again for the tips! Below is the current configuration, please
> > suggest changes if any.
> >
> > ---- HAProxy ----
> >
> > frontend galera-fe
> >         mode tcp
> >         bind 10.231.4.112:3306
> >         use_backend galera-be
> >
> > backend galera-be
> >         balance source
> >         mode tcp
> >         option tcpka
> >         option mysql-check user haproxy
> >         server galera-0 10.231.4.36:3306 check
> >         server galera-1 10.231.4.37:3306 check
> >         server galera-2 10.231.4.38:3306 check
> >
> > ---- Keepalived ----
> >
> > vrrp_script check_backend {
> >     script "killall -0 haproxy"
> >     weight -20
> >     interval 2
> >     rise 2
> >     fall 2
> > }
> >
> > vrrp_instance DB_0 {
> >   state MASTER  # BACKUP on others
> >   priority 100
> >   interface enp1s0
> >   virtual_router_id 50
> >   advert_int 1
> >   unicast_peer {
> >         10.231.4.87 # Relevant on others
> >         10.231.4.88 # Relevant on others
> >   }
> >   virtual_ipaddress {
> >     10.231.4.112/24
> >   }
> >   track_script {
> >       check_backend
> >   }
> > }
> >
> > Best Regards,
> > Jayanth
> >
> > On Tue, May 3, 2022 at 12:33 PM Ivan Kudryavtsev <iv...@bw-sw.com> wrote:
> >
> > > Sounds cool,
> > >
> > > Just ensure that in any failure case (db, haproxy, OS or hardware
> crash)
> > > all the Management servers are switched to the same Galera instance,
> > > otherwise, this could lead to operational problems.
> > > Also, backups are still mandatory, recommend doing them from one of
> > > Galera's hot-swap nodes, not from the main operational node.
> > >
> > > Best wishes, Ivan
> > >
> > > On Tue, May 3, 2022 at 9:53 AM Jayanth Reddy <
> jayanthreddy5666@gmail.com
> > >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > >     Thank you. Have set up MariaDB Galera Cluster with the required
> > > HAProxy
> > > > configuration with MYSQL health checks.  Everything is working fine.
> > > >
> > > > On Mon, May 2, 2022 at 10:48 AM Ivan Kudryavtsev <iv...@bw-sw.com>
> > wrote:
> > > >
> > > > > Hi, I use MariaDB Galera cluster.
> > > > >
> > > > > But you have to pin all the CS management to the same galera node
> to
> > > make
> > > > > cloudstack transactioned operations work correctly. HAproxy or
> shared
> > > > > common ip solve that.
> > > > >
> > > > > пн, 2 мая 2022 г., 7:34 AM Jayanth Reddy <
> jayanthreddy5666@gmail.com
> > >:
> > > > >
> > > > > > Hello guys,
> > > > > >
> > > > > >     How are you doing database High Availability? Any inputs on
> DB
> > > > > > Clustering and CloudStack configuration would really help me.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Database High Availability

Posted by Ivan Kudryavtsev <iv...@bw-sw.com>.
Well,

You would better consult with real-life mysql experts, as for me, I
referred to great severalnines.com articles like:
https://severalnines.com/resources/database-management-tutorials/galera-cluster-mysql-tutorial
https://severalnines.com/database-blog/avoiding-deadlocks-galera-setting-haproxy-single-node-writes-and-multi-node-reads

Just take a look for best practices there.



On Tue, May 3, 2022 at 10:18 AM Jayanth Reddy <ja...@gmail.com>
wrote:

> Hi,
>
>  Thanks again for the tips! Below is the current configuration, please
> suggest changes if any.
>
> ---- HAProxy ----
>
> frontend galera-fe
>         mode tcp
>         bind 10.231.4.112:3306
>         use_backend galera-be
>
> backend galera-be
>         balance source
>         mode tcp
>         option tcpka
>         option mysql-check user haproxy
>         server galera-0 10.231.4.36:3306 check
>         server galera-1 10.231.4.37:3306 check
>         server galera-2 10.231.4.38:3306 check
>
> ---- Keepalived ----
>
> vrrp_script check_backend {
>     script "killall -0 haproxy"
>     weight -20
>     interval 2
>     rise 2
>     fall 2
> }
>
> vrrp_instance DB_0 {
>   state MASTER  # BACKUP on others
>   priority 100
>   interface enp1s0
>   virtual_router_id 50
>   advert_int 1
>   unicast_peer {
>         10.231.4.87 # Relevant on others
>         10.231.4.88 # Relevant on others
>   }
>   virtual_ipaddress {
>     10.231.4.112/24
>   }
>   track_script {
>       check_backend
>   }
> }
>
> Best Regards,
> Jayanth
>
> On Tue, May 3, 2022 at 12:33 PM Ivan Kudryavtsev <iv...@bw-sw.com> wrote:
>
> > Sounds cool,
> >
> > Just ensure that in any failure case (db, haproxy, OS or hardware crash)
> > all the Management servers are switched to the same Galera instance,
> > otherwise, this could lead to operational problems.
> > Also, backups are still mandatory, recommend doing them from one of
> > Galera's hot-swap nodes, not from the main operational node.
> >
> > Best wishes, Ivan
> >
> > On Tue, May 3, 2022 at 9:53 AM Jayanth Reddy <jayanthreddy5666@gmail.com
> >
> > wrote:
> >
> > > Hi,
> > >
> > >     Thank you. Have set up MariaDB Galera Cluster with the required
> > HAProxy
> > > configuration with MYSQL health checks.  Everything is working fine.
> > >
> > > On Mon, May 2, 2022 at 10:48 AM Ivan Kudryavtsev <iv...@bw-sw.com>
> wrote:
> > >
> > > > Hi, I use MariaDB Galera cluster.
> > > >
> > > > But you have to pin all the CS management to the same galera node to
> > make
> > > > cloudstack transactioned operations work correctly. HAproxy or shared
> > > > common ip solve that.
> > > >
> > > > пн, 2 мая 2022 г., 7:34 AM Jayanth Reddy <jayanthreddy5666@gmail.com
> >:
> > > >
> > > > > Hello guys,
> > > > >
> > > > >     How are you doing database High Availability? Any inputs on DB
> > > > > Clustering and CloudStack configuration would really help me.
> > > > >
> > > >
> > >
> >
>

Re: Database High Availability

Posted by Jayanth Reddy <ja...@gmail.com>.
Hi,

 Thanks again for the tips! Below is the current configuration, please
suggest changes if any.

---- HAProxy ----

frontend galera-fe
        mode tcp
        bind 10.231.4.112:3306
        use_backend galera-be

backend galera-be
        balance source
        mode tcp
        option tcpka
        option mysql-check user haproxy
        server galera-0 10.231.4.36:3306 check
        server galera-1 10.231.4.37:3306 check
        server galera-2 10.231.4.38:3306 check

---- Keepalived ----

vrrp_script check_backend {
    script "killall -0 haproxy"
    weight -20
    interval 2
    rise 2
    fall 2
}

vrrp_instance DB_0 {
  state MASTER  # BACKUP on others
  priority 100
  interface enp1s0
  virtual_router_id 50
  advert_int 1
  unicast_peer {
        10.231.4.87 # Relevant on others
        10.231.4.88 # Relevant on others
  }
  virtual_ipaddress {
    10.231.4.112/24
  }
  track_script {
      check_backend
  }
}

Best Regards,
Jayanth

On Tue, May 3, 2022 at 12:33 PM Ivan Kudryavtsev <iv...@bw-sw.com> wrote:

> Sounds cool,
>
> Just ensure that in any failure case (db, haproxy, OS or hardware crash)
> all the Management servers are switched to the same Galera instance,
> otherwise, this could lead to operational problems.
> Also, backups are still mandatory, recommend doing them from one of
> Galera's hot-swap nodes, not from the main operational node.
>
> Best wishes, Ivan
>
> On Tue, May 3, 2022 at 9:53 AM Jayanth Reddy <ja...@gmail.com>
> wrote:
>
> > Hi,
> >
> >     Thank you. Have set up MariaDB Galera Cluster with the required
> HAProxy
> > configuration with MYSQL health checks.  Everything is working fine.
> >
> > On Mon, May 2, 2022 at 10:48 AM Ivan Kudryavtsev <iv...@bw-sw.com> wrote:
> >
> > > Hi, I use MariaDB Galera cluster.
> > >
> > > But you have to pin all the CS management to the same galera node to
> make
> > > cloudstack transactioned operations work correctly. HAproxy or shared
> > > common ip solve that.
> > >
> > > пн, 2 мая 2022 г., 7:34 AM Jayanth Reddy <ja...@gmail.com>:
> > >
> > > > Hello guys,
> > > >
> > > >     How are you doing database High Availability? Any inputs on DB
> > > > Clustering and CloudStack configuration would really help me.
> > > >
> > >
> >
>

Re: Database High Availability

Posted by Ivan Kudryavtsev <iv...@bw-sw.com>.
Sounds cool,

Just ensure that in any failure case (db, haproxy, OS or hardware crash)
all the Management servers are switched to the same Galera instance,
otherwise, this could lead to operational problems.
Also, backups are still mandatory, recommend doing them from one of
Galera's hot-swap nodes, not from the main operational node.

Best wishes, Ivan

On Tue, May 3, 2022 at 9:53 AM Jayanth Reddy <ja...@gmail.com>
wrote:

> Hi,
>
>     Thank you. Have set up MariaDB Galera Cluster with the required HAProxy
> configuration with MYSQL health checks.  Everything is working fine.
>
> On Mon, May 2, 2022 at 10:48 AM Ivan Kudryavtsev <iv...@bw-sw.com> wrote:
>
> > Hi, I use MariaDB Galera cluster.
> >
> > But you have to pin all the CS management to the same galera node to make
> > cloudstack transactioned operations work correctly. HAproxy or shared
> > common ip solve that.
> >
> > пн, 2 мая 2022 г., 7:34 AM Jayanth Reddy <ja...@gmail.com>:
> >
> > > Hello guys,
> > >
> > >     How are you doing database High Availability? Any inputs on DB
> > > Clustering and CloudStack configuration would really help me.
> > >
> >
>

Re: Database High Availability

Posted by Jayanth Reddy <ja...@gmail.com>.
Hi,

    Thank you. Have set up MariaDB Galera Cluster with the required HAProxy
configuration with MYSQL health checks.  Everything is working fine.

On Mon, May 2, 2022 at 10:48 AM Ivan Kudryavtsev <iv...@bw-sw.com> wrote:

> Hi, I use MariaDB Galera cluster.
>
> But you have to pin all the CS management to the same galera node to make
> cloudstack transactioned operations work correctly. HAproxy or shared
> common ip solve that.
>
> пн, 2 мая 2022 г., 7:34 AM Jayanth Reddy <ja...@gmail.com>:
>
> > Hello guys,
> >
> >     How are you doing database High Availability? Any inputs on DB
> > Clustering and CloudStack configuration would really help me.
> >
>

Re: Database High Availability

Posted by Ivan Kudryavtsev <iv...@bw-sw.com>.
Hi, I use MariaDB Galera cluster.

But you have to pin all the CS management to the same galera node to make
cloudstack transactioned operations work correctly. HAproxy or shared
common ip solve that.

пн, 2 мая 2022 г., 7:34 AM Jayanth Reddy <ja...@gmail.com>:

> Hello guys,
>
>     How are you doing database High Availability? Any inputs on DB
> Clustering and CloudStack configuration would really help me.
>