You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ofbiz.apache.org by Jacques Le Roux <ja...@les7arts.com> on 2017/03/08 16:36:08 UTC
Re: database replication
Hi Paul,
I just wonder, are you using Amazon RDS in production? And if so how went the last big outage?
Thanks
Jacques
Le 17/01/2017 � 21:42, Paul Mandeltort a �crit :
> Honestly, probably not. There are several HA Postgres hosts nowadays that are very cost-effective. Amazon RDS is one, for example, and just a few clicks and boom you have a replicated fail-over Postgres instance available to you for practically nothing. Set up good backup scheme and you have a bulletproof database infrastructure.
>
> OFBiz has lots of areas that need attention (back-end user experience\u2026.), but scaling and cloud-hosting databases is a problem that\u2019s being solved already by many other folks, so no sense in re-inventing the wheel here.
>
> Bandwidth costs will continue to fall across the entire world. The cost of supporting custom/odd-ball features in OFBiz that the whole community doesn\u2019t need will continue to rise as the software ages and increases technical debt.
>
> \u2014P
>
>> On Jan 17, 2017, at 12:47 PM, Bahaa Alamood <ba...@arcdigitalsolutionsandconsultancy.com> wrote:
>>
>> Hi Paul,
>>
>> Thanks for the advice :), but this is not the point here. The point is having such a solution will provide you with the clustered environment that will make your important business data always stored in more than one machine (I think you know how important that is) We can forget about the remote office scenario and think cloud or vps type situation where you need this redundancy, you will see that this scenario is very valuable. Another way to think about this when this remote office is located in parts of the world where a Mbps cost over 300 USD a month then this option is a valuable option, is it not?
>>
>>
>> On 1/17/2017 1:10 PM, Paul Mandeltort wrote:
>>> How expensive would it be to just upgrade the office\u2019s internet connectivity? With my business hat on, that will almost always be MUCH cheaper than investing in a corner case of development that the rest of the community isn\u2019t using anyway.
>>>
>>> Take a look at the new generation of microwave data links available - you can now securely link sites at 1gbps+ point-to-point over 5-10 miles with 99.999% uptime with modest hardware investments (which against are always cheaper than trying to re-architect OFBiz).
>>>
>>> \u2014P
>>>
>>>> On Jan 17, 2017, at 10:15 AM, Bahaa Alamood <ba...@arcdigitalsolutionsandconsultancy.com> wrote:
>>>>
>>>> Hi Jaques,
>>>>
>>>> thanks for the tip, I have seen this post before. I tried it today and it seems to do the trick for the sequences. I think I do not need the rest of the setup in the post because I want to use the brd which sync the database.
>>>>
>>>> I have a questions regarding the sync. So i have a scenario as an example where one user on one server changes a record and I have another user on another server changed the same record. would the system be able to handle this when they sync? will I see both changes as history?
>>>>
>>>>
>>>> On 1/17/2017 4:29 AM, Jacques Le Roux wrote:
>>>>> The now removed POS component used a solution for similar cases. This solution still exists and is reliable on a LAN ("less" on Internet)
>>>>>
>>>>> You can find the documentation at https://cwiki.apache.org/confluence/display/OFBIZ/Sync+Setup+Notes+and+Example
>>>>>
>>>>> Disclaimer: it's not very easy to understand and use...
>>>>>
>>>>> HTH
>>>>>
>>>>> Jacques
>>>>>
>>>>>
>>>>> Le 17/01/2017 � 00:01, Bahaa Alamood a �crit :
>>>>>> Hello All,
>>>>>>
>>>>>> We have a situation that we need to have more than 2 servers of ofbiz running, server A, B, and C. The scenario is like this
>>>>>>
>>>>>> 1. Server A is the main server and it is online all the time with its own database. some people do connect to that and make changes to the data (create, delete, modify)
>>>>>>
>>>>>> 2. Server B is a server in one office (own database) and they do not have a reliable internet connection so it goes offline some times while the users in this office continue to use their local ofbiz
>>>>>>
>>>>>> 3. Server C is the same as server B with its own database as well
>>>>>>
>>>>>> I have been looking at the bdr from 2nd Quadrant https://2ndquadrant.com/en/resources/bdr/ to do the data replicationamong the servers. I realize that this could create conflicts in the primary keys of many things in the system. So I looked at SequenceUtil.java and I can see if I change the stagger in getNextSeqId from 1 to 2 in one of the systems I can avoid this conflict in two of them, but what about the third one? also in the above scenario what else I need to look out for other than the sequences that might create conflicts?
>>>>>>
>
>
Re: database replication
Posted by Jacques Le Roux <ja...@les7arts.com>.
Thanks Paul!
Jacques
Le 08/03/2017 � 17:49, Paul Mandeltort a �crit :
> We actually haven\u2019t migrated our live setup yet - we are finishing up a few other projects including getting some docker stuff sorted out to make our IT headaches a lot easier.
>
> The outage only affected the S3 service which is separate from RDS, and as far as I know there was no data loss, just an outage where the services were offline for a bit.
>
> We got a prototype setup on my hardware with Rancher container management going and it\u2019s REALLY promising for a proper OFbiz Dev->Test->Live workflow with almost no headaches and distributed contractors/developers. I will shared our findings with the group once we get it all working.
>
> RDS does provide automated backup services so even if the outage did affect it I don\u2019t think there would have been any data loss.
>
> \u2014P
>
>
>> On Mar 8, 2017, at 11:36 AM, Jacques Le Roux <ja...@les7arts.com> wrote:
>>
>> Hi Paul,
>>
>> I just wonder, are you using Amazon RDS in production? And if so how went the last big outage?
>>
>> Thanks
>>
>> Jacques
>>
>>
>> Le 17/01/2017 � 21:42, Paul Mandeltort a �crit :
>>> Honestly, probably not. There are several HA Postgres hosts nowadays that are very cost-effective. Amazon RDS is one, for example, and just a few clicks and boom you have a replicated fail-over Postgres instance available to you for practically nothing. Set up good backup scheme and you have a bulletproof database infrastructure.
>>>
>>> OFBiz has lots of areas that need attention (back-end user experience\u2026.), but scaling and cloud-hosting databases is a problem that\u2019s being solved already by many other folks, so no sense in re-inventing the wheel here.
>>>
>>> Bandwidth costs will continue to fall across the entire world. The cost of supporting custom/odd-ball features in OFBiz that the whole community doesn\u2019t need will continue to rise as the software ages and increases technical debt.
>>>
>>> \u2014P
>>>
>>>> On Jan 17, 2017, at 12:47 PM, Bahaa Alamood <ba...@arcdigitalsolutionsandconsultancy.com> wrote:
>>>>
>>>> Hi Paul,
>>>>
>>>> Thanks for the advice :), but this is not the point here. The point is having such a solution will provide you with the clustered environment that will make your important business data always stored in more than one machine (I think you know how important that is) We can forget about the remote office scenario and think cloud or vps type situation where you need this redundancy, you will see that this scenario is very valuable. Another way to think about this when this remote office is located in parts of the world where a Mbps cost over 300 USD a month then this option is a valuable option, is it not?
>>>>
>>>>
>>>> On 1/17/2017 1:10 PM, Paul Mandeltort wrote:
>>>>> How expensive would it be to just upgrade the office\u2019s internet connectivity? With my business hat on, that will almost always be MUCH cheaper than investing in a corner case of development that the rest of the community isn\u2019t using anyway.
>>>>>
>>>>> Take a look at the new generation of microwave data links available - you can now securely link sites at 1gbps+ point-to-point over 5-10 miles with 99.999% uptime with modest hardware investments (which against are always cheaper than trying to re-architect OFBiz).
>>>>>
>>>>> \u2014P
>>>>>
>>>>>> On Jan 17, 2017, at 10:15 AM, Bahaa Alamood <ba...@arcdigitalsolutionsandconsultancy.com> wrote:
>>>>>>
>>>>>> Hi Jaques,
>>>>>>
>>>>>> thanks for the tip, I have seen this post before. I tried it today and it seems to do the trick for the sequences. I think I do not need the rest of the setup in the post because I want to use the brd which sync the database.
>>>>>>
>>>>>> I have a questions regarding the sync. So i have a scenario as an example where one user on one server changes a record and I have another user on another server changed the same record. would the system be able to handle this when they sync? will I see both changes as history?
>>>>>>
>>>>>>
>>>>>> On 1/17/2017 4:29 AM, Jacques Le Roux wrote:
>>>>>>> The now removed POS component used a solution for similar cases. This solution still exists and is reliable on a LAN ("less" on Internet)
>>>>>>>
>>>>>>> You can find the documentation at https://cwiki.apache.org/confluence/display/OFBIZ/Sync+Setup+Notes+and+Example
>>>>>>>
>>>>>>> Disclaimer: it's not very easy to understand and use...
>>>>>>>
>>>>>>> HTH
>>>>>>>
>>>>>>> Jacques
>>>>>>>
>>>>>>>
>>>>>>> Le 17/01/2017 � 00:01, Bahaa Alamood a �crit :
>>>>>>>> Hello All,
>>>>>>>>
>>>>>>>> We have a situation that we need to have more than 2 servers of ofbiz running, server A, B, and C. The scenario is like this
>>>>>>>>
>>>>>>>> 1. Server A is the main server and it is online all the time with its own database. some people do connect to that and make changes to the data (create, delete, modify)
>>>>>>>>
>>>>>>>> 2. Server B is a server in one office (own database) and they do not have a reliable internet connection so it goes offline some times while the users in this office continue to use their local ofbiz
>>>>>>>>
>>>>>>>> 3. Server C is the same as server B with its own database as well
>>>>>>>>
>>>>>>>> I have been looking at the bdr from 2nd Quadrant https://2ndquadrant.com/en/resources/bdr/ to do the data replicationamong the servers. I realize that this could create conflicts in the primary keys of many things in the system. So I looked at SequenceUtil.java and I can see if I change the stagger in getNextSeqId from 1 to 2 in one of the systems I can avoid this conflict in two of them, but what about the third one? also in the above scenario what else I need to look out for other than the sequences that might create conflicts?
>>>>>>>>
>>>
>
>
Re: database replication
Posted by Paul Mandeltort <pa...@marcospec.com>.
We actually haven’t migrated our live setup yet - we are finishing up a few other projects including getting some docker stuff sorted out to make our IT headaches a lot easier.
The outage only affected the S3 service which is separate from RDS, and as far as I know there was no data loss, just an outage where the services were offline for a bit.
We got a prototype setup on my hardware with Rancher container management going and it’s REALLY promising for a proper OFbiz Dev->Test->Live workflow with almost no headaches and distributed contractors/developers. I will shared our findings with the group once we get it all working.
RDS does provide automated backup services so even if the outage did affect it I don’t think there would have been any data loss.
—P
> On Mar 8, 2017, at 11:36 AM, Jacques Le Roux <ja...@les7arts.com> wrote:
>
> Hi Paul,
>
> I just wonder, are you using Amazon RDS in production? And if so how went the last big outage?
>
> Thanks
>
> Jacques
>
>
> Le 17/01/2017 à 21:42, Paul Mandeltort a écrit :
>> Honestly, probably not. There are several HA Postgres hosts nowadays that are very cost-effective. Amazon RDS is one, for example, and just a few clicks and boom you have a replicated fail-over Postgres instance available to you for practically nothing. Set up good backup scheme and you have a bulletproof database infrastructure.
>>
>> OFBiz has lots of areas that need attention (back-end user experience….), but scaling and cloud-hosting databases is a problem that’s being solved already by many other folks, so no sense in re-inventing the wheel here.
>>
>> Bandwidth costs will continue to fall across the entire world. The cost of supporting custom/odd-ball features in OFBiz that the whole community doesn’t need will continue to rise as the software ages and increases technical debt.
>>
>> —P
>>
>>> On Jan 17, 2017, at 12:47 PM, Bahaa Alamood <ba...@arcdigitalsolutionsandconsultancy.com> wrote:
>>>
>>> Hi Paul,
>>>
>>> Thanks for the advice :), but this is not the point here. The point is having such a solution will provide you with the clustered environment that will make your important business data always stored in more than one machine (I think you know how important that is) We can forget about the remote office scenario and think cloud or vps type situation where you need this redundancy, you will see that this scenario is very valuable. Another way to think about this when this remote office is located in parts of the world where a Mbps cost over 300 USD a month then this option is a valuable option, is it not?
>>>
>>>
>>> On 1/17/2017 1:10 PM, Paul Mandeltort wrote:
>>>> How expensive would it be to just upgrade the office’s internet connectivity? With my business hat on, that will almost always be MUCH cheaper than investing in a corner case of development that the rest of the community isn’t using anyway.
>>>>
>>>> Take a look at the new generation of microwave data links available - you can now securely link sites at 1gbps+ point-to-point over 5-10 miles with 99.999% uptime with modest hardware investments (which against are always cheaper than trying to re-architect OFBiz).
>>>>
>>>> —P
>>>>
>>>>> On Jan 17, 2017, at 10:15 AM, Bahaa Alamood <ba...@arcdigitalsolutionsandconsultancy.com> wrote:
>>>>>
>>>>> Hi Jaques,
>>>>>
>>>>> thanks for the tip, I have seen this post before. I tried it today and it seems to do the trick for the sequences. I think I do not need the rest of the setup in the post because I want to use the brd which sync the database.
>>>>>
>>>>> I have a questions regarding the sync. So i have a scenario as an example where one user on one server changes a record and I have another user on another server changed the same record. would the system be able to handle this when they sync? will I see both changes as history?
>>>>>
>>>>>
>>>>> On 1/17/2017 4:29 AM, Jacques Le Roux wrote:
>>>>>> The now removed POS component used a solution for similar cases. This solution still exists and is reliable on a LAN ("less" on Internet)
>>>>>>
>>>>>> You can find the documentation at https://cwiki.apache.org/confluence/display/OFBIZ/Sync+Setup+Notes+and+Example
>>>>>>
>>>>>> Disclaimer: it's not very easy to understand and use...
>>>>>>
>>>>>> HTH
>>>>>>
>>>>>> Jacques
>>>>>>
>>>>>>
>>>>>> Le 17/01/2017 à 00:01, Bahaa Alamood a écrit :
>>>>>>> Hello All,
>>>>>>>
>>>>>>> We have a situation that we need to have more than 2 servers of ofbiz running, server A, B, and C. The scenario is like this
>>>>>>>
>>>>>>> 1. Server A is the main server and it is online all the time with its own database. some people do connect to that and make changes to the data (create, delete, modify)
>>>>>>>
>>>>>>> 2. Server B is a server in one office (own database) and they do not have a reliable internet connection so it goes offline some times while the users in this office continue to use their local ofbiz
>>>>>>>
>>>>>>> 3. Server C is the same as server B with its own database as well
>>>>>>>
>>>>>>> I have been looking at the bdr from 2nd Quadrant https://2ndquadrant.com/en/resources/bdr/ to do the data replicationamong the servers. I realize that this could create conflicts in the primary keys of many things in the system. So I looked at SequenceUtil.java and I can see if I change the stagger in getNextSeqId from 1 to 2 in one of the systems I can avoid this conflict in two of them, but what about the third one? also in the above scenario what else I need to look out for other than the sequences that might create conflicts?
>>>>>>>
>>
>>
>