Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

All,

Currently, under the environment of cloudstack with multiple regions, each
region has its own management server running with a separate database. So
if we want to support multiple regions and provide one point of entry for a
customer, we need to duplicate domain/account/user information of that
customer to all of the databases of regions the customer accesses, which
will cause data discrepancies when users update those data independently in
each management server.

So I'd like to provide a way to sync up the data using the messaging system
introduced in 4.1.0. Using the events from each management server, updates
from each region can be propagated to the rest regions and they can be
executed accordingly.

I hope you guys have a chance to think about this and give some feedbacks
if interested.
Thanks in advance.
Alex Ough

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[David Grizzanti <da...@sungard.com>]

Alex,

That sounds reasonable. Can we add the change to the API create calls for domain/account/user to the scope of this work?

Thanks

-- 
David Grizzanti
Software Engineer
Sungard Availability Services
e: david.grizzanti@sungard.com
w: 215.446.1431
c: 570.575.0315

On February 19, 2014 at 6:32:07 PM, Alex Ough (alex.ough@sungard.com) wrote:

Hi Dave,

"resources have different uuids in different regions even if they are identical" is not a restriction but a normal case because maintaining a same uuid can be difficult in some cases like when 2 different users create the same resource in different regions at the same time.

But it will not be an issue if you still want to maintain the same UUIDs because the resource map table will have the same uuid for each resource across all regions. The only thing to be changed is to set the resource UUID in the API create calls to send the create requests to remote regions during the real time synchronization.

Let me know if this is not clear.
Thanks
Alex Ough


On Wed, Feb 19, 2014 at 1:47 PM, David Grizzanti <da...@sungard.com> wrote:
Hi Alex,

One thing I wanted to ask about/mention on this was regarding the restriction you have mentioned in the wiki on "resources have different uuids in different regions even if they are identical".  We discussed this a bit offline, but I think it would be beneficial to allow for the the UUIDs to be carried over to the other regions when you're replicating resources. I'm not sure how others feel about this feature, but I know that in our discussions we will need this feature if we are to rely on the sync to create domains/accounts/user across all the regions as the UUID is in the identifying factor of uniqueness.

Let me know your thoughts on this and whether or not this can be added, at least as an optional item if needed.

Thanks

-- 
David Grizzanti
Software Engineer
Sungard Availability Services
e: david.grizzanti@sungard.com
w: 215.446.1431
c: 570.575.0315

On February 6, 2014 at 3:19:52 PM, Alex Ough (alex.ough@sungard.com) wrote:

Hi Chiradeep,
Thanks for your reply.

The change is just to add timestamps when record has been changed to decide
the time order when a same resource has been changed independently in
different regions.
The changes are minimum and additions, so I don't think they will cause any
side effects.

Thanks
Alex Ough


On Thu, Feb 6, 2014 at 1:35 PM, Chiradeep Vittal <
Chiradeep.Vittal@citrix.com> wrote:

> I am uncomfortable with changes to GenericDaoBase. Was this really
> necessary? This feature was supposed to be "outside" CloudStack as much as
> possible and optional. Yet it touches the most sensitive code in CloudStack.
>
> From: Alex Ough <al...@sungard.com>
> Date: Thursday, February 6, 2014 6:29 AM
> To: "dev@cloudstack.apache.org" <de...@cloudstack.apache.org>
> Cc: Chip Childers <ch...@gmail.org>, Daan Hoogland <
> daan.hoogland@gmail.com>, Chiradeep Vittal <ch...@citrix.com>,
> Kishan Kavala <Ki...@citrix.com>
> Subject: Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
>
> All,
>
> I just sent a review request, so please take a look at it and let me
> know if you have any comments/suggests.
>
> https://reviews.apache.org/r/17790/
>
> Thanks
> Alex Ough
>
>
> On Mon, Jan 13, 2014 at 11:17 AM, Alex Ough <al...@sungard.com> wrote:
>
>> All,
>>
>> I'd like to have some suggestion about 2 things related with this.
>>
>> 1. The 'Full Scan' management
>> Now, I set it running every time a user logs in to the UI, but I think it
>> will be necessary to make it run with some interval also.
>> But I'm not familiar with the config file, so can anyone give some
>> directions how to manage the time interval in the config file and the best
>> way to run it with the time interval?
>>
>> 2. Repository of regions with their login information.
>> To send/receive requests to/from other regions using API interfaces, we
>> need the region information including login info of each region.
>> I was planning to use a table as a repository, but I think it is better
>> to store it in the config file to make the access a little lighter.
>> Any recommendation on this?
>>
>> Your reply with directions & comments will be very appreciated.
>> Thanks
>> Alex Ough
>>
>>
>> On Wed, Jan 8, 2014 at 2:17 PM, Alex Ough <al...@sungard.com> wrote:
>>
>>> All,
>>>
>>> A little bit of updates after a long vacation,
>>> I'm currently creating automated test scripts that randomly
>>> create/delete/update domain/account/user objects in random regions to
>>> trigger the sync-up and full scans regularly.
>>> Once they are completed, I'll post it in the github also and submit the
>>> review requests for this implementation.
>>>
>>> Let me know if you have any comments.
>>> Thanks
>>> Alex Ough
>>>
>>>
>>> On Wed, Dec 18, 2013 at 3:39 PM, Alex Ough <al...@sungard.com>wrote:
>>>
>>>> All,
>>>>
>>>> I updated the wiki after some logic changes, so please review them,
>>>> especially "Full Scan", which is newly introduced.
>>>>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
>>>>
>>>> And I implemented this functionality in Java and you can get the pull
>>>> request of it here.
>>>> (This does not include the 'full scan' yet and I'm currently working
>>>> on this to finalize.)
>>>> https://github.com/alexoughsg/Albatross/pull/1
>>>>
>>>> Especially, I really want to have your review on the "Full Scan" logic
>>>> to confirm if it does not miss any cases.
>>>> Thanks for your interest and your feedback will be very helpful.
>>>> Alex Ough
>>>>
>>>> On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough <al...@sungard.com>
>>>> wrote:
>>>> > Good point, Chiradeep,
>>>> >
>>>> > I'm not sure if you reviewed my design doc in the wiki, but my design
>>>> is to
>>>> > just skip any actions for target resources that already took place by
>>>> any
>>>> > means.
>>>> > But the issue is when conflict actions in the same resources (like
>>>> create &
>>>> > delete the same users) are enqueued in reversed orders, which is
>>>> hopefully
>>>> > rare.
>>>> >
>>>> > And to support consistency in the AP system, I'd like to provide a
>>>> full sync
>>>> > up, which will sync up all data in all region servers by selecting a
>>>> region
>>>> > as a master and force its data to other regions.
>>>> >
>>>> > Let me know what you think.
>>>> > Thanks
>>>> > Alex Ough
>>>> >
>>>> >
>>>> > On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal
>>>> > <Ch...@citrix.com> wrote:
>>>> >>
>>>> >> Missed this one. In a single region, the CloudStack DB is the master
>>>> for
>>>> >> most operations. If the infra is not in the state the DB says it
>>>> should
>>>> >> be, generally the approach is to whack it and make it conform. For
>>>> some
>>>> >> exceptions (live migration/related use cases are exceptions) the DB
>>>> is the
>>>> >> slave -- the point is that the inconsistency that inevitably arise
>>>> in an
>>>> >> AP system need a conflict resolution system. In a single region, the
>>>> >> default is to assume that the MySQL DB is correct and handle other
>>>> cases
>>>> >> carefully.
>>>> >>
>>>> >> In a multi-region case, there is no master: disable an account in one
>>>> >> region, and it may not propagate to the other regions for many
>>>> hours/days.
>>>> >> You could add a user in one region and then add the same user in a
>>>> >> different region and conflict before the sync happens.
>>>> >>
>>>> >> This is of course not a problem unique to CloudStack -- people pay
>>>> mucho
>>>> >> dinero for Global AD/LDAP sync. I don't think this is a problem for
>>>> >> CloudStack core to solve: I support the event-based mechanism for
>>>> those
>>>> >> who want this facility.
>>>> >>
>>>> >> Distributed systems are hard and research continues to try and make
>>>> >> building these systems easier, but there are very few solutions for
>>>> global
>>>> >> state synchronization (Google Spanner comes to mind)
>>>> >>
>>>> >> --
>>>> >> Chiradeep
>>>> >>
>>>> >>
>>>> >> On 11/8/13 4:53 PM, "Chip Childers" <ch...@gmail.com> wrote:
>>>> >>
>>>> >> >We are already (generally) AP for most infra changes really. I'd
>>>> use that
>>>> >> >model. Eventual consistency is better in this scenario.
>>>> >> >
>>>> >> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal
>>>> >> >><Ch...@citrix.com> wrote:
>>>> >> >>
>>>> >> >> I'd also like to highlight that it isn't a trivial problem.
>>>> >> >> Let's say there's 3 regions: this means there are 3 copies of the
>>>> user
>>>> >> >> database that are geographically separated by network links that
>>>> fail
>>>> >> >> quite often (orders of magnitude more than intra-DC networks).
>>>> >> >>
>>>> >> >> Here we run into the consequences of the CAP theorem [1].
>>>> >> >> We can either have a CP or AP system: either approach makes some
>>>> >> >>tradeoffs:
>>>> >> >> 1. If we run a AP system, then the challenge is to resolve
>>>> conflicting
>>>> >> >> updates
>>>> >> >> 2. If we run a CP system, then the challenge is to detect
>>>> partitions
>>>> >> >> reliably and disallow updates during partitions.
>>>> >> >>
>>>> >> >> [1] http://en.wikipedia.org/wiki/CAP_theorem
>>>> >> >>
>>>> >> >>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org>
>>>> wrote:
>>>> >> >>>
>>>> >> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
>>>> >> >>> <Ch...@citrix.com> wrote:
>>>> >> >>>> It may be an admin burden, but it has to be optional. There are
>>>> other
>>>> >> >>>> ways
>>>> >> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).
>>>> >> >>>> A lot of service providers who run cloudstack have their own
>>>> user
>>>> >> >>>> database
>>>> >> >>>> / portal. In their implementations the CloudStack database is
>>>> not the
>>>> >> >>>> master source of user records, but a slave.
>>>> >> >>>
>>>> >> >>> +1 to it being optional.
>>>> >> >>
>>>> >>
>>>> >>
>>>> >
>>>>
>>>
>>>
>>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

Hi Dave,

"resources have different uuids in different regions even if they are
identical" is not a restriction but a normal case because maintaining a
same uuid can be difficult in some cases like when 2 different users create
the same resource in different regions at the same time.

But it will not be an issue if you still want to maintain the same UUIDs
because the resource map table will have the same uuid for each resource
across all regions. The only thing to be changed is to set the resource
UUID in the API create calls to send the create requests to remote regions
during the real time synchronization.

Let me know if this is not clear.
Thanks
Alex Ough


On Wed, Feb 19, 2014 at 1:47 PM, David Grizzanti <
david.grizzanti@sungard.com> wrote:

> Hi Alex,
>
> One thing I wanted to ask about/mention on this was regarding the
> restriction you have mentioned in the wiki on "resources have different
> uuids in different regions even if they are identical".  We discussed this
> a bit offline, but I think it would be beneficial to allow for the the
> UUIDs to be carried over to the other regions when you're replicating
> resources. I'm not sure how others feel about this feature, but I know that
> in our discussions we will need this feature if we are to rely on the sync
> to create domains/accounts/user across all the regions as the UUID is in
> the identifying factor of uniqueness.
>
> Let me know your thoughts on this and whether or not this can be added, at
> least as an optional item if needed.
>
> Thanks
>
> --
> David Grizzanti
> Software Engineer
> Sungard Availability Services
> e: david.grizzanti@sungard.com
> w: 215.446.1431
> c: 570.575.0315
>
> On February 6, 2014 at 3:19:52 PM, Alex Ough (alex.ough@sungard.com<//...@sungard.com>)
> wrote:
>
> Hi Chiradeep,
> Thanks for your reply.
>
> The change is just to add timestamps when record has been changed to
> decide
> the time order when a same resource has been changed independently in
> different regions.
> The changes are minimum and additions, so I don't think they will cause
> any
> side effects.
>
> Thanks
> Alex Ough
>
>
> On Thu, Feb 6, 2014 at 1:35 PM, Chiradeep Vittal <
> Chiradeep.Vittal@citrix.com> wrote:
>
> > I am uncomfortable with changes to GenericDaoBase. Was this really
> > necessary? This feature was supposed to be "outside" CloudStack as much
> as
> > possible and optional. Yet it touches the most sensitive code in
> CloudStack.
> >
> > From: Alex Ough <al...@sungard.com>
> > Date: Thursday, February 6, 2014 6:29 AM
> > To: "dev@cloudstack.apache.org" <de...@cloudstack.apache.org>
> > Cc: Chip Childers <ch...@gmail.org>, Daan Hoogland <
> > daan.hoogland@gmail.com>, Chiradeep Vittal <ch...@citrix.com>,
>
> > Kishan Kavala <Ki...@citrix.com>
> > Subject: Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple
> Regions
> >
> > All,
> >
> > I just sent a review request, so please take a look at it and let me
> > know if you have any comments/suggests.
> >
> > https://reviews.apache.org/r/17790/
> >
> > Thanks
> > Alex Ough
> >
> >
> > On Mon, Jan 13, 2014 at 11:17 AM, Alex Ough <al...@sungard.com>
> wrote:
> >
> >> All,
> >>
> >> I'd like to have some suggestion about 2 things related with this.
> >>
> >> 1. The 'Full Scan' management
> >> Now, I set it running every time a user logs in to the UI, but I think
> it
> >> will be necessary to make it run with some interval also.
> >> But I'm not familiar with the config file, so can anyone give some
> >> directions how to manage the time interval in the config file and the
> best
> >> way to run it with the time interval?
> >>
> >> 2. Repository of regions with their login information.
> >> To send/receive requests to/from other regions using API interfaces, we
> >> need the region information including login info of each region.
> >> I was planning to use a table as a repository, but I think it is better
> >> to store it in the config file to make the access a little lighter.
> >> Any recommendation on this?
> >>
> >> Your reply with directions & comments will be very appreciated.
> >> Thanks
> >> Alex Ough
> >>
> >>
> >> On Wed, Jan 8, 2014 at 2:17 PM, Alex Ough <al...@sungard.com>
> wrote:
> >>
> >>> All,
> >>>
> >>> A little bit of updates after a long vacation,
> >>> I'm currently creating automated test scripts that randomly
> >>> create/delete/update domain/account/user objects in random regions to
> >>> trigger the sync-up and full scans regularly.
> >>> Once they are completed, I'll post it in the github also and submit
> the
> >>> review requests for this implementation.
> >>>
> >>> Let me know if you have any comments.
> >>> Thanks
> >>> Alex Ough
> >>>
> >>>
> >>> On Wed, Dec 18, 2013 at 3:39 PM, Alex Ough <al...@sungard.com>wrote:
>
> >>>
> >>>> All,
> >>>>
> >>>> I updated the wiki after some logic changes, so please review them,
> >>>> especially "Full Scan", which is newly introduced.
> >>>>
> >>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
> >>>>
> >>>> And I implemented this functionality in Java and you can get the pull
> >>>> request of it here.
> >>>> (This does not include the 'full scan' yet and I'm currently working
> >>>> on this to finalize.)
> >>>> https://github.com/alexoughsg/Albatross/pull/1
> >>>>
> >>>> Especially, I really want to have your review on the "Full Scan"
> logic
> >>>> to confirm if it does not miss any cases.
> >>>> Thanks for your interest and your feedback will be very helpful.
> >>>> Alex Ough
> >>>>
> >>>> On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough <al...@sungard.com>
> >>>> wrote:
> >>>> > Good point, Chiradeep,
> >>>> >
> >>>> > I'm not sure if you reviewed my design doc in the wiki, but my
> design
> >>>> is to
> >>>> > just skip any actions for target resources that already took place
> by
> >>>> any
> >>>> > means.
> >>>> > But the issue is when conflict actions in the same resources (like
> >>>> create &
> >>>> > delete the same users) are enqueued in reversed orders, which is
> >>>> hopefully
> >>>> > rare.
> >>>> >
> >>>> > And to support consistency in the AP system, I'd like to provide a
> >>>> full sync
> >>>> > up, which will sync up all data in all region servers by selecting
> a
> >>>> region
> >>>> > as a master and force its data to other regions.
> >>>> >
> >>>> > Let me know what you think.
> >>>> > Thanks
> >>>> > Alex Ough
> >>>> >
> >>>> >
> >>>> > On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal
> >>>> > <Ch...@citrix.com> wrote:
> >>>> >>
> >>>> >> Missed this one. In a single region, the CloudStack DB is the
> master
> >>>> for
> >>>> >> most operations. If the infra is not in the state the DB says it
> >>>> should
> >>>> >> be, generally the approach is to whack it and make it conform. For
> >>>> some
> >>>> >> exceptions (live migration/related use cases are exceptions) the
> DB
> >>>> is the
> >>>> >> slave -- the point is that the inconsistency that inevitably arise
> >>>> in an
> >>>> >> AP system need a conflict resolution system. In a single region,
> the
> >>>> >> default is to assume that the MySQL DB is correct and handle other
> >>>> cases
> >>>> >> carefully.
> >>>> >>
> >>>> >> In a multi-region case, there is no master: disable an account in
> one
> >>>> >> region, and it may not propagate to the other regions for many
> >>>> hours/days.
> >>>> >> You could add a user in one region and then add the same user in a
> >>>> >> different region and conflict before the sync happens.
> >>>> >>
> >>>> >> This is of course not a problem unique to CloudStack -- people pay
> >>>> mucho
> >>>> >> dinero for Global AD/LDAP sync. I don't think this is a problem
> for
> >>>> >> CloudStack core to solve: I support the event-based mechanism for
> >>>> those
> >>>> >> who want this facility.
> >>>> >>
> >>>> >> Distributed systems are hard and research continues to try and
> make
> >>>> >> building these systems easier, but there are very few solutions
> for
> >>>> global
> >>>> >> state synchronization (Google Spanner comes to mind)
> >>>> >>
> >>>> >> --
> >>>> >> Chiradeep
> >>>> >>
> >>>> >>
> >>>> >> On 11/8/13 4:53 PM, "Chip Childers" <ch...@gmail.com>
> wrote:
> >>>> >>
> >>>> >> >We are already (generally) AP for most infra changes really. I'd
> >>>> use that
> >>>> >> >model. Eventual consistency is better in this scenario.
> >>>> >> >
> >>>> >> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal
> >>>> >> >><Ch...@citrix.com> wrote:
> >>>> >> >>
> >>>> >> >> I'd also like to highlight that it isn't a trivial problem.
> >>>> >> >> Let's say there's 3 regions: this means there are 3 copies of
> the
> >>>> user
> >>>> >> >> database that are geographically separated by network links
> that
> >>>> fail
> >>>> >> >> quite often (orders of magnitude more than intra-DC networks).
> >>>> >> >>
> >>>> >> >> Here we run into the consequences of the CAP theorem [1].
> >>>> >> >> We can either have a CP or AP system: either approach makes
> some
> >>>> >> >>tradeoffs:
> >>>> >> >> 1. If we run a AP system, then the challenge is to resolve
> >>>> conflicting
> >>>> >> >> updates
> >>>> >> >> 2. If we run a CP system, then the challenge is to detect
> >>>> partitions
> >>>> >> >> reliably and disallow updates during partitions.
> >>>> >> >>
> >>>> >> >> [1] http://en.wikipedia.org/wiki/CAP_theorem
> >>>> >> >>
> >>>> >> >>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org>
>
> >>>> wrote:
> >>>> >> >>>
> >>>> >> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
> >>>> >> >>> <Ch...@citrix.com> wrote:
> >>>> >> >>>> It may be an admin burden, but it has to be optional. There
> are
> >>>> other
> >>>> >> >>>> ways
> >>>> >> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).
> >>>> >> >>>> A lot of service providers who run cloudstack have their own
> >>>> user
> >>>> >> >>>> database
> >>>> >> >>>> / portal. In their implementations the CloudStack database is
> >>>> not the
> >>>> >> >>>> master source of user records, but a slave.
> >>>> >> >>>
> >>>> >> >>> +1 to it being optional.
> >>>> >> >>
> >>>> >>
> >>>> >>
> >>>> >
> >>>>
> >>>
> >>>
> >>
> >
>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[David Grizzanti <da...@sungard.com>]

Hi Alex,

One thing I wanted to ask about/mention on this was regarding the restriction you have mentioned in the wiki on "resources have different uuids in different regions even if they are identical".  We discussed this a bit offline, but I think it would be beneficial to allow for the the UUIDs to be carried over to the other regions when you're replicating resources. I'm not sure how others feel about this feature, but I know that in our discussions we will need this feature if we are to rely on the sync to create domains/accounts/user across all the regions as the UUID is in the identifying factor of uniqueness.

Let me know your thoughts on this and whether or not this can be added, at least as an optional item if needed.

Thanks

-- 
David Grizzanti
Software Engineer
Sungard Availability Services
e: david.grizzanti@sungard.com
w: 215.446.1431
c: 570.575.0315

On February 6, 2014 at 3:19:52 PM, Alex Ough (alex.ough@sungard.com) wrote:

Hi Chiradeep,  
Thanks for your reply.  

The change is just to add timestamps when record has been changed to decide  
the time order when a same resource has been changed independently in  
different regions.  
The changes are minimum and additions, so I don't think they will cause any  
side effects.  

Thanks  
Alex Ough  


On Thu, Feb 6, 2014 at 1:35 PM, Chiradeep Vittal <  
Chiradeep.Vittal@citrix.com> wrote:  

> I am uncomfortable with changes to GenericDaoBase. Was this really  
> necessary? This feature was supposed to be "outside" CloudStack as much as  
> possible and optional. Yet it touches the most sensitive code in CloudStack.  
>  
> From: Alex Ough <al...@sungard.com>  
> Date: Thursday, February 6, 2014 6:29 AM  
> To: "dev@cloudstack.apache.org" <de...@cloudstack.apache.org>  
> Cc: Chip Childers <ch...@gmail.org>, Daan Hoogland <  
> daan.hoogland@gmail.com>, Chiradeep Vittal <ch...@citrix.com>,  
> Kishan Kavala <Ki...@citrix.com>  
> Subject: Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions  
>  
> All,  
>  
> I just sent a review request, so please take a look at it and let me  
> know if you have any comments/suggests.  
>  
> https://reviews.apache.org/r/17790/  
>  
> Thanks  
> Alex Ough  
>  
>  
> On Mon, Jan 13, 2014 at 11:17 AM, Alex Ough <al...@sungard.com> wrote:  
>  
>> All,  
>>  
>> I'd like to have some suggestion about 2 things related with this.  
>>  
>> 1. The 'Full Scan' management  
>> Now, I set it running every time a user logs in to the UI, but I think it  
>> will be necessary to make it run with some interval also.  
>> But I'm not familiar with the config file, so can anyone give some  
>> directions how to manage the time interval in the config file and the best  
>> way to run it with the time interval?  
>>  
>> 2. Repository of regions with their login information.  
>> To send/receive requests to/from other regions using API interfaces, we  
>> need the region information including login info of each region.  
>> I was planning to use a table as a repository, but I think it is better  
>> to store it in the config file to make the access a little lighter.  
>> Any recommendation on this?  
>>  
>> Your reply with directions & comments will be very appreciated.  
>> Thanks  
>> Alex Ough  
>>  
>>  
>> On Wed, Jan 8, 2014 at 2:17 PM, Alex Ough <al...@sungard.com> wrote:  
>>  
>>> All,  
>>>  
>>> A little bit of updates after a long vacation,  
>>> I'm currently creating automated test scripts that randomly  
>>> create/delete/update domain/account/user objects in random regions to  
>>> trigger the sync-up and full scans regularly.  
>>> Once they are completed, I'll post it in the github also and submit the  
>>> review requests for this implementation.  
>>>  
>>> Let me know if you have any comments.  
>>> Thanks  
>>> Alex Ough  
>>>  
>>>  
>>> On Wed, Dec 18, 2013 at 3:39 PM, Alex Ough <al...@sungard.com>wrote:  
>>>  
>>>> All,  
>>>>  
>>>> I updated the wiki after some logic changes, so please review them,  
>>>> especially "Full Scan", which is newly introduced.  
>>>>  
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions  
>>>>  
>>>> And I implemented this functionality in Java and you can get the pull  
>>>> request of it here.  
>>>> (This does not include the 'full scan' yet and I'm currently working  
>>>> on this to finalize.)  
>>>> https://github.com/alexoughsg/Albatross/pull/1  
>>>>  
>>>> Especially, I really want to have your review on the "Full Scan" logic  
>>>> to confirm if it does not miss any cases.  
>>>> Thanks for your interest and your feedback will be very helpful.  
>>>> Alex Ough  
>>>>  
>>>> On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough <al...@sungard.com>  
>>>> wrote:  
>>>> > Good point, Chiradeep,  
>>>> >  
>>>> > I'm not sure if you reviewed my design doc in the wiki, but my design  
>>>> is to  
>>>> > just skip any actions for target resources that already took place by  
>>>> any  
>>>> > means.  
>>>> > But the issue is when conflict actions in the same resources (like  
>>>> create &  
>>>> > delete the same users) are enqueued in reversed orders, which is  
>>>> hopefully  
>>>> > rare.  
>>>> >  
>>>> > And to support consistency in the AP system, I'd like to provide a  
>>>> full sync  
>>>> > up, which will sync up all data in all region servers by selecting a  
>>>> region  
>>>> > as a master and force its data to other regions.  
>>>> >  
>>>> > Let me know what you think.  
>>>> > Thanks  
>>>> > Alex Ough  
>>>> >  
>>>> >  
>>>> > On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal  
>>>> > <Ch...@citrix.com> wrote:  
>>>> >>  
>>>> >> Missed this one. In a single region, the CloudStack DB is the master  
>>>> for  
>>>> >> most operations. If the infra is not in the state the DB says it  
>>>> should  
>>>> >> be, generally the approach is to whack it and make it conform. For  
>>>> some  
>>>> >> exceptions (live migration/related use cases are exceptions) the DB  
>>>> is the  
>>>> >> slave -- the point is that the inconsistency that inevitably arise  
>>>> in an  
>>>> >> AP system need a conflict resolution system. In a single region, the  
>>>> >> default is to assume that the MySQL DB is correct and handle other  
>>>> cases  
>>>> >> carefully.  
>>>> >>  
>>>> >> In a multi-region case, there is no master: disable an account in one  
>>>> >> region, and it may not propagate to the other regions for many  
>>>> hours/days.  
>>>> >> You could add a user in one region and then add the same user in a  
>>>> >> different region and conflict before the sync happens.  
>>>> >>  
>>>> >> This is of course not a problem unique to CloudStack -- people pay  
>>>> mucho  
>>>> >> dinero for Global AD/LDAP sync. I don't think this is a problem for  
>>>> >> CloudStack core to solve: I support the event-based mechanism for  
>>>> those  
>>>> >> who want this facility.  
>>>> >>  
>>>> >> Distributed systems are hard and research continues to try and make  
>>>> >> building these systems easier, but there are very few solutions for  
>>>> global  
>>>> >> state synchronization (Google Spanner comes to mind)  
>>>> >>  
>>>> >> --  
>>>> >> Chiradeep  
>>>> >>  
>>>> >>  
>>>> >> On 11/8/13 4:53 PM, "Chip Childers" <ch...@gmail.com> wrote:  
>>>> >>  
>>>> >> >We are already (generally) AP for most infra changes really. I'd  
>>>> use that  
>>>> >> >model. Eventual consistency is better in this scenario.  
>>>> >> >  
>>>> >> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal  
>>>> >> >><Ch...@citrix.com> wrote:  
>>>> >> >>  
>>>> >> >> I'd also like to highlight that it isn't a trivial problem.  
>>>> >> >> Let's say there's 3 regions: this means there are 3 copies of the  
>>>> user  
>>>> >> >> database that are geographically separated by network links that  
>>>> fail  
>>>> >> >> quite often (orders of magnitude more than intra-DC networks).  
>>>> >> >>  
>>>> >> >> Here we run into the consequences of the CAP theorem [1].  
>>>> >> >> We can either have a CP or AP system: either approach makes some  
>>>> >> >>tradeoffs:  
>>>> >> >> 1. If we run a AP system, then the challenge is to resolve  
>>>> conflicting  
>>>> >> >> updates  
>>>> >> >> 2. If we run a CP system, then the challenge is to detect  
>>>> partitions  
>>>> >> >> reliably and disallow updates during partitions.  
>>>> >> >>  
>>>> >> >> [1] http://en.wikipedia.org/wiki/CAP_theorem  
>>>> >> >>  
>>>> >> >>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org>  
>>>> wrote:  
>>>> >> >>>  
>>>> >> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal  
>>>> >> >>> <Ch...@citrix.com> wrote:  
>>>> >> >>>> It may be an admin burden, but it has to be optional. There are  
>>>> other  
>>>> >> >>>> ways  
>>>> >> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).  
>>>> >> >>>> A lot of service providers who run cloudstack have their own  
>>>> user  
>>>> >> >>>> database  
>>>> >> >>>> / portal. In their implementations the CloudStack database is  
>>>> not the  
>>>> >> >>>> master source of user records, but a slave.  
>>>> >> >>>  
>>>> >> >>> +1 to it being optional.  
>>>> >> >>  
>>>> >>  
>>>> >>  
>>>> >  
>>>>  
>>>  
>>>  
>>  
>  

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

Hi Chiradeep,
Thanks for your reply.

The change is just to add timestamps when record has been changed to decide
the time order when a same resource has been changed independently in
different regions.
The changes are minimum and additions, so I don't think they will cause any
side effects.

Thanks
Alex Ough


On Thu, Feb 6, 2014 at 1:35 PM, Chiradeep Vittal <
Chiradeep.Vittal@citrix.com> wrote:

>  I am uncomfortable with changes to GenericDaoBase. Was this really
> necessary? This feature was supposed to be "outside" CloudStack as much as
> possible and optional. Yet it touches the most sensitive code in CloudStack.
>
>   From: Alex Ough <al...@sungard.com>
> Date: Thursday, February 6, 2014 6:29 AM
> To: "dev@cloudstack.apache.org" <de...@cloudstack.apache.org>
> Cc: Chip Childers <ch...@gmail.org>, Daan Hoogland <
> daan.hoogland@gmail.com>, Chiradeep Vittal <ch...@citrix.com>,
> Kishan Kavala <Ki...@citrix.com>
> Subject: Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions
>
>   All,
>
>  I just sent a review request, so please take a look at it and let me
> know if you have any comments/suggests.
>
>  https://reviews.apache.org/r/17790/
>
>  Thanks
> Alex Ough
>
>
> On Mon, Jan 13, 2014 at 11:17 AM, Alex Ough <al...@sungard.com> wrote:
>
>> All,
>>
>>  I'd like to have some suggestion about 2 things related with this.
>>
>>  1. The 'Full Scan' management
>> Now, I set it running every time a user logs in to the UI, but I think it
>> will be necessary to make it run with some interval also.
>> But I'm not familiar with the config file, so can anyone give some
>> directions how to manage the time interval in the config file and the best
>> way to run it with the time interval?
>>
>>  2. Repository of regions with their login information.
>> To send/receive requests to/from other regions using API interfaces, we
>> need the region information including login info of each region.
>> I was planning to use a table as a repository, but I think it is better
>> to store it in the config file to make the access a little lighter.
>> Any recommendation on this?
>>
>>  Your reply with directions & comments will be very appreciated.
>> Thanks
>>  Alex Ough
>>
>>
>> On Wed, Jan 8, 2014 at 2:17 PM, Alex Ough <al...@sungard.com> wrote:
>>
>>> All,
>>>
>>>  A little bit of updates after a long vacation,
>>> I'm currently creating automated test scripts that randomly
>>> create/delete/update domain/account/user objects in random regions to
>>> trigger the sync-up and full scans regularly.
>>> Once they are completed, I'll post it in the github also and submit the
>>> review requests for this implementation.
>>>
>>>  Let me know if you have any comments.
>>> Thanks
>>>  Alex Ough
>>>
>>>
>>> On Wed, Dec 18, 2013 at 3:39 PM, Alex Ough <al...@sungard.com>wrote:
>>>
>>>> All,
>>>>
>>>> I updated the wiki after some logic changes, so please review them,
>>>> especially "Full Scan", which is newly introduced.
>>>>
>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
>>>>
>>>> And I implemented this functionality in Java and you can get the pull
>>>> request of it here.
>>>> (This does not include the 'full scan' yet and I'm currently working
>>>> on this to finalize.)
>>>> https://github.com/alexoughsg/Albatross/pull/1
>>>>
>>>> Especially, I really want to have your review on the "Full Scan" logic
>>>> to confirm if it does not miss any cases.
>>>> Thanks for your interest and your feedback will be very helpful.
>>>> Alex Ough
>>>>
>>>> On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough <al...@sungard.com>
>>>> wrote:
>>>> > Good point, Chiradeep,
>>>> >
>>>> > I'm not sure if you reviewed my design doc in the wiki, but my design
>>>> is to
>>>> > just skip any actions for target resources that already took place by
>>>> any
>>>> > means.
>>>> > But the issue is when conflict actions in the same resources (like
>>>> create &
>>>> > delete the same users) are enqueued in reversed orders, which is
>>>> hopefully
>>>> > rare.
>>>> >
>>>> > And to support consistency in the AP system, I'd like to provide a
>>>> full sync
>>>> > up, which will sync up all data in all region servers by selecting a
>>>> region
>>>> > as a master and force its data to other regions.
>>>> >
>>>> > Let me know what you think.
>>>> > Thanks
>>>> > Alex Ough
>>>> >
>>>> >
>>>> > On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal
>>>> > <Ch...@citrix.com> wrote:
>>>> >>
>>>> >> Missed this one. In a single region, the CloudStack DB is the master
>>>> for
>>>> >> most operations. If the infra is not in the state the DB says it
>>>> should
>>>> >> be, generally the approach is to whack it and make it conform. For
>>>> some
>>>> >> exceptions (live migration/related use cases are exceptions) the DB
>>>> is the
>>>> >> slave -- the point is that the inconsistency that inevitably arise
>>>> in an
>>>> >> AP system need a conflict resolution system. In a single region, the
>>>> >> default is to assume that the MySQL DB is correct and handle other
>>>> cases
>>>> >> carefully.
>>>> >>
>>>> >> In a multi-region case, there is no master: disable an account in one
>>>> >> region, and it may not propagate to the other regions for many
>>>> hours/days.
>>>> >> You could add a user in one region and then add the same user in a
>>>> >> different region and conflict before the sync happens.
>>>> >>
>>>> >> This is of course not a problem unique to CloudStack -- people pay
>>>> mucho
>>>> >> dinero for Global AD/LDAP sync. I don't think this is a problem for
>>>> >> CloudStack core to solve: I support the event-based mechanism for
>>>> those
>>>> >> who want this facility.
>>>> >>
>>>> >> Distributed systems are hard and research continues to try and make
>>>> >> building these systems easier, but there are very few solutions for
>>>> global
>>>> >> state synchronization (Google Spanner comes to mind)
>>>> >>
>>>> >> --
>>>> >> Chiradeep
>>>> >>
>>>> >>
>>>> >> On 11/8/13 4:53 PM, "Chip Childers" <ch...@gmail.com> wrote:
>>>> >>
>>>> >> >We are already (generally) AP for most infra changes really. I'd
>>>> use that
>>>> >> >model. Eventual consistency is better in this scenario.
>>>> >> >
>>>> >> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal
>>>> >> >><Ch...@citrix.com> wrote:
>>>> >> >>
>>>> >> >> I'd also like to highlight that it isn't a trivial problem.
>>>> >> >> Let's say there's 3 regions: this means there are 3 copies of the
>>>> user
>>>> >> >> database that are geographically separated by network links that
>>>> fail
>>>> >> >> quite often (orders of magnitude more than intra-DC networks).
>>>> >> >>
>>>> >> >> Here we run into the consequences of the CAP theorem [1].
>>>> >> >> We can either have a CP or AP system: either approach makes some
>>>> >> >>tradeoffs:
>>>> >> >> 1. If we run a AP system, then the challenge is to resolve
>>>> conflicting
>>>> >> >> updates
>>>> >> >> 2. If we run a CP system, then the challenge is to detect
>>>> partitions
>>>> >> >> reliably and disallow updates during partitions.
>>>> >> >>
>>>> >> >> [1] http://en.wikipedia.org/wiki/CAP_theorem
>>>> >> >>
>>>> >> >>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org>
>>>> wrote:
>>>> >> >>>
>>>> >> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
>>>> >> >>> <Ch...@citrix.com> wrote:
>>>> >> >>>> It may be an admin burden, but it has to be optional. There are
>>>> other
>>>> >> >>>> ways
>>>> >> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).
>>>> >> >>>> A lot of service providers who run cloudstack have their own
>>>> user
>>>> >> >>>> database
>>>> >> >>>> / portal. In their implementations the CloudStack database is
>>>> not the
>>>> >> >>>> master source of user records, but a slave.
>>>> >> >>>
>>>> >> >>> +1 to it being optional.
>>>> >> >>
>>>> >>
>>>> >>
>>>> >
>>>>
>>>
>>>
>>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Chiradeep Vittal <Ch...@citrix.com>]

I am uncomfortable with changes to GenericDaoBase. Was this really necessary? This feature was supposed to be "outside" CloudStack as much as possible and optional. Yet it touches the most sensitive code in CloudStack.

From: Alex Ough <al...@sungard.com>>
Date: Thursday, February 6, 2014 6:29 AM
To: "dev@cloudstack.apache.org<ma...@cloudstack.apache.org>" <de...@cloudstack.apache.org>>
Cc: Chip Childers <ch...@gmail.org>>, Daan Hoogland <da...@gmail.com>>, Chiradeep Vittal <ch...@citrix.com>>, Kishan Kavala <Ki...@citrix.com>>
Subject: Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

All,

I just sent a review request, so please take a look at it and let me know if you have any comments/suggests.

https://reviews.apache.org/r/17790/

Thanks
Alex Ough


On Mon, Jan 13, 2014 at 11:17 AM, Alex Ough <al...@sungard.com>> wrote:
All,

I'd like to have some suggestion about 2 things related with this.

1. The 'Full Scan' management
Now, I set it running every time a user logs in to the UI, but I think it will be necessary to make it run with some interval also.
But I'm not familiar with the config file, so can anyone give some directions how to manage the time interval in the config file and the best way to run it with the time interval?

2. Repository of regions with their login information.
To send/receive requests to/from other regions using API interfaces, we need the region information including login info of each region.
I was planning to use a table as a repository, but I think it is better to store it in the config file to make the access a little lighter.
Any recommendation on this?

Your reply with directions & comments will be very appreciated.
Thanks
Alex Ough


On Wed, Jan 8, 2014 at 2:17 PM, Alex Ough <al...@sungard.com>> wrote:
All,

A little bit of updates after a long vacation,
I'm currently creating automated test scripts that randomly create/delete/update domain/account/user objects in random regions to trigger the sync-up and full scans regularly.
Once they are completed, I'll post it in the github also and submit the review requests for this implementation.

Let me know if you have any comments.
Thanks
Alex Ough


On Wed, Dec 18, 2013 at 3:39 PM, Alex Ough <al...@sungard.com>> wrote:
All,

I updated the wiki after some logic changes, so please review them,
especially "Full Scan", which is newly introduced.
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions

And I implemented this functionality in Java and you can get the pull
request of it here.
(This does not include the 'full scan' yet and I'm currently working
on this to finalize.)
https://github.com/alexoughsg/Albatross/pull/1

Especially, I really want to have your review on the "Full Scan" logic
to confirm if it does not miss any cases.
Thanks for your interest and your feedback will be very helpful.
Alex Ough

On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough <al...@sungard.com>> wrote:
> Good point, Chiradeep,
>
> I'm not sure if you reviewed my design doc in the wiki, but my design is to
> just skip any actions for target resources that already took place by any
> means.
> But the issue is when conflict actions in the same resources (like create &
> delete the same users) are enqueued in reversed orders, which is hopefully
> rare.
>
> And to support consistency in the AP system, I'd like to provide a full sync
> up, which will sync up all data in all region servers by selecting a region
> as a master and force its data to other regions.
>
> Let me know what you think.
> Thanks
> Alex Ough
>
>
> On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal
> <Ch...@citrix.com>> wrote:
>>
>> Missed this one. In a single region, the CloudStack DB is the master for
>> most operations. If the infra is not in the state the DB says it should
>> be, generally the approach is to whack it and make it conform. For some
>> exceptions (live migration/related use cases are exceptions) the DB is the
>> slave -- the point is that the inconsistency that inevitably arise in an
>> AP system need a conflict resolution system. In a single region, the
>> default is to assume that the MySQL DB is correct and handle other cases
>> carefully.
>>
>> In a multi-region case, there is no master: disable an account in one
>> region, and it may not propagate to the other regions for many hours/days.
>> You could add a user in one region and then add the same user in a
>> different region and conflict before the sync happens.
>>
>> This is of course not a problem unique to CloudStack -- people pay mucho
>> dinero for Global AD/LDAP sync. I don't think this is a problem for
>> CloudStack core to solve: I support the event-based mechanism for those
>> who want this facility.
>>
>> Distributed systems are hard and research continues to try and make
>> building these systems easier, but there are very few solutions for global
>> state synchronization (Google Spanner comes to mind)
>>
>> --
>> Chiradeep
>>
>>
>> On 11/8/13 4:53 PM, "Chip Childers" <ch...@gmail.com>> wrote:
>>
>> >We are already (generally) AP for most infra changes really. I'd use that
>> >model. Eventual consistency is better in this scenario.
>> >
>> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal
>> >><Ch...@citrix.com>> wrote:
>> >>
>> >> I'd also like to highlight that it isn't a trivial problem.
>> >> Let's say there's 3 regions: this means there are 3 copies of the user
>> >> database that are geographically separated by network links that fail
>> >> quite often (orders of magnitude more than intra-DC networks).
>> >>
>> >> Here we run into the consequences of the CAP theorem [1].
>> >> We can either have a CP or AP system: either approach makes some
>> >>tradeoffs:
>> >> 1. If we run a AP system, then the challenge is to resolve conflicting
>> >> updates
>> >> 2. If we run a CP system, then the challenge is to detect partitions
>> >> reliably and disallow updates during partitions.
>> >>
>> >> [1] http://en.wikipedia.org/wiki/CAP_theorem
>> >>
>> >>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org>> wrote:
>> >>>
>> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
>> >>> <Ch...@citrix.com>> wrote:
>> >>>> It may be an admin burden, but it has to be optional. There are other
>> >>>> ways
>> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).
>> >>>> A lot of service providers who run cloudstack have their own user
>> >>>> database
>> >>>> / portal. In their implementations the CloudStack database is not the
>> >>>> master source of user records, but a slave.
>> >>>
>> >>> +1 to it being optional.
>> >>
>>
>>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

All,

I just sent a review request, so please take a look at it and let me know
if you have any comments/suggests.

https://reviews.apache.org/r/17790/

Thanks
Alex Ough


On Mon, Jan 13, 2014 at 11:17 AM, Alex Ough <al...@sungard.com> wrote:

> All,
>
> I'd like to have some suggestion about 2 things related with this.
>
> 1. The 'Full Scan' management
> Now, I set it running every time a user logs in to the UI, but I think it
> will be necessary to make it run with some interval also.
> But I'm not familiar with the config file, so can anyone give some
> directions how to manage the time interval in the config file and the best
> way to run it with the time interval?
>
> 2. Repository of regions with their login information.
> To send/receive requests to/from other regions using API interfaces, we
> need the region information including login info of each region.
> I was planning to use a table as a repository, but I think it is better to
> store it in the config file to make the access a little lighter.
> Any recommendation on this?
>
> Your reply with directions & comments will be very appreciated.
> Thanks
> Alex Ough
>
>
> On Wed, Jan 8, 2014 at 2:17 PM, Alex Ough <al...@sungard.com> wrote:
>
>> All,
>>
>> A little bit of updates after a long vacation,
>> I'm currently creating automated test scripts that randomly
>> create/delete/update domain/account/user objects in random regions to
>> trigger the sync-up and full scans regularly.
>> Once they are completed, I'll post it in the github also and submit the
>> review requests for this implementation.
>>
>> Let me know if you have any comments.
>> Thanks
>> Alex Ough
>>
>>
>> On Wed, Dec 18, 2013 at 3:39 PM, Alex Ough <al...@sungard.com> wrote:
>>
>>> All,
>>>
>>> I updated the wiki after some logic changes, so please review them,
>>> especially "Full Scan", which is newly introduced.
>>>
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
>>>
>>> And I implemented this functionality in Java and you can get the pull
>>> request of it here.
>>> (This does not include the 'full scan' yet and I'm currently working
>>> on this to finalize.)
>>> https://github.com/alexoughsg/Albatross/pull/1
>>>
>>> Especially, I really want to have your review on the "Full Scan" logic
>>> to confirm if it does not miss any cases.
>>> Thanks for your interest and your feedback will be very helpful.
>>> Alex Ough
>>>
>>> On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough <al...@sungard.com>
>>> wrote:
>>> > Good point, Chiradeep,
>>> >
>>> > I'm not sure if you reviewed my design doc in the wiki, but my design
>>> is to
>>> > just skip any actions for target resources that already took place by
>>> any
>>> > means.
>>> > But the issue is when conflict actions in the same resources (like
>>> create &
>>> > delete the same users) are enqueued in reversed orders, which is
>>> hopefully
>>> > rare.
>>> >
>>> > And to support consistency in the AP system, I'd like to provide a
>>> full sync
>>> > up, which will sync up all data in all region servers by selecting a
>>> region
>>> > as a master and force its data to other regions.
>>> >
>>> > Let me know what you think.
>>> > Thanks
>>> > Alex Ough
>>> >
>>> >
>>> > On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal
>>> > <Ch...@citrix.com> wrote:
>>> >>
>>> >> Missed this one. In a single region, the CloudStack DB is the master
>>> for
>>> >> most operations. If the infra is not in the state the DB says it
>>> should
>>> >> be, generally the approach is to whack it and make it conform. For
>>> some
>>> >> exceptions (live migration/related use cases are exceptions) the DB
>>> is the
>>> >> slave -- the point is that the inconsistency that inevitably arise in
>>> an
>>> >> AP system need a conflict resolution system. In a single region, the
>>> >> default is to assume that the MySQL DB is correct and handle other
>>> cases
>>> >> carefully.
>>> >>
>>> >> In a multi-region case, there is no master: disable an account in one
>>> >> region, and it may not propagate to the other regions for many
>>> hours/days.
>>> >> You could add a user in one region and then add the same user in a
>>> >> different region and conflict before the sync happens.
>>> >>
>>> >> This is of course not a problem unique to CloudStack -- people pay
>>> mucho
>>> >> dinero for Global AD/LDAP sync. I don't think this is a problem for
>>> >> CloudStack core to solve: I support the event-based mechanism for
>>> those
>>> >> who want this facility.
>>> >>
>>> >> Distributed systems are hard and research continues to try and make
>>> >> building these systems easier, but there are very few solutions for
>>> global
>>> >> state synchronization (Google Spanner comes to mind)
>>> >>
>>> >> --
>>> >> Chiradeep
>>> >>
>>> >>
>>> >> On 11/8/13 4:53 PM, "Chip Childers" <ch...@gmail.com> wrote:
>>> >>
>>> >> >We are already (generally) AP for most infra changes really. I'd use
>>> that
>>> >> >model. Eventual consistency is better in this scenario.
>>> >> >
>>> >> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal
>>> >> >><Ch...@citrix.com> wrote:
>>> >> >>
>>> >> >> I'd also like to highlight that it isn't a trivial problem.
>>> >> >> Let's say there's 3 regions: this means there are 3 copies of the
>>> user
>>> >> >> database that are geographically separated by network links that
>>> fail
>>> >> >> quite often (orders of magnitude more than intra-DC networks).
>>> >> >>
>>> >> >> Here we run into the consequences of the CAP theorem [1].
>>> >> >> We can either have a CP or AP system: either approach makes some
>>> >> >>tradeoffs:
>>> >> >> 1. If we run a AP system, then the challenge is to resolve
>>> conflicting
>>> >> >> updates
>>> >> >> 2. If we run a CP system, then the challenge is to detect
>>> partitions
>>> >> >> reliably and disallow updates during partitions.
>>> >> >>
>>> >> >> [1] http://en.wikipedia.org/wiki/CAP_theorem
>>> >> >>
>>> >> >>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org>
>>> wrote:
>>> >> >>>
>>> >> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
>>> >> >>> <Ch...@citrix.com> wrote:
>>> >> >>>> It may be an admin burden, but it has to be optional. There are
>>> other
>>> >> >>>> ways
>>> >> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).
>>> >> >>>> A lot of service providers who run cloudstack have their own user
>>> >> >>>> database
>>> >> >>>> / portal. In their implementations the CloudStack database is
>>> not the
>>> >> >>>> master source of user records, but a slave.
>>> >> >>>
>>> >> >>> +1 to it being optional.
>>> >> >>
>>> >>
>>> >>
>>> >
>>>
>>
>>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

All,

I'd like to have some suggestion about 2 things related with this.

1. The 'Full Scan' management
Now, I set it running every time a user logs in to the UI, but I think it
will be necessary to make it run with some interval also.
But I'm not familiar with the config file, so can anyone give some
directions how to manage the time interval in the config file and the best
way to run it with the time interval?

2. Repository of regions with their login information.
To send/receive requests to/from other regions using API interfaces, we
need the region information including login info of each region.
I was planning to use a table as a repository, but I think it is better to
store it in the config file to make the access a little lighter.
Any recommendation on this?

Your reply with directions & comments will be very appreciated.
Thanks
Alex Ough


On Wed, Jan 8, 2014 at 2:17 PM, Alex Ough <al...@sungard.com> wrote:

> All,
>
> A little bit of updates after a long vacation,
> I'm currently creating automated test scripts that randomly
> create/delete/update domain/account/user objects in random regions to
> trigger the sync-up and full scans regularly.
> Once they are completed, I'll post it in the github also and submit the
> review requests for this implementation.
>
> Let me know if you have any comments.
> Thanks
> Alex Ough
>
>
> On Wed, Dec 18, 2013 at 3:39 PM, Alex Ough <al...@sungard.com> wrote:
>
>> All,
>>
>> I updated the wiki after some logic changes, so please review them,
>> especially "Full Scan", which is newly introduced.
>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
>>
>> And I implemented this functionality in Java and you can get the pull
>> request of it here.
>> (This does not include the 'full scan' yet and I'm currently working
>> on this to finalize.)
>> https://github.com/alexoughsg/Albatross/pull/1
>>
>> Especially, I really want to have your review on the "Full Scan" logic
>> to confirm if it does not miss any cases.
>> Thanks for your interest and your feedback will be very helpful.
>> Alex Ough
>>
>> On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough <al...@sungard.com> wrote:
>> > Good point, Chiradeep,
>> >
>> > I'm not sure if you reviewed my design doc in the wiki, but my design
>> is to
>> > just skip any actions for target resources that already took place by
>> any
>> > means.
>> > But the issue is when conflict actions in the same resources (like
>> create &
>> > delete the same users) are enqueued in reversed orders, which is
>> hopefully
>> > rare.
>> >
>> > And to support consistency in the AP system, I'd like to provide a full
>> sync
>> > up, which will sync up all data in all region servers by selecting a
>> region
>> > as a master and force its data to other regions.
>> >
>> > Let me know what you think.
>> > Thanks
>> > Alex Ough
>> >
>> >
>> > On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal
>> > <Ch...@citrix.com> wrote:
>> >>
>> >> Missed this one. In a single region, the CloudStack DB is the master
>> for
>> >> most operations. If the infra is not in the state the DB says it should
>> >> be, generally the approach is to whack it and make it conform. For some
>> >> exceptions (live migration/related use cases are exceptions) the DB is
>> the
>> >> slave -- the point is that the inconsistency that inevitably arise in
>> an
>> >> AP system need a conflict resolution system. In a single region, the
>> >> default is to assume that the MySQL DB is correct and handle other
>> cases
>> >> carefully.
>> >>
>> >> In a multi-region case, there is no master: disable an account in one
>> >> region, and it may not propagate to the other regions for many
>> hours/days.
>> >> You could add a user in one region and then add the same user in a
>> >> different region and conflict before the sync happens.
>> >>
>> >> This is of course not a problem unique to CloudStack -- people pay
>> mucho
>> >> dinero for Global AD/LDAP sync. I don't think this is a problem for
>> >> CloudStack core to solve: I support the event-based mechanism for those
>> >> who want this facility.
>> >>
>> >> Distributed systems are hard and research continues to try and make
>> >> building these systems easier, but there are very few solutions for
>> global
>> >> state synchronization (Google Spanner comes to mind)
>> >>
>> >> --
>> >> Chiradeep
>> >>
>> >>
>> >> On 11/8/13 4:53 PM, "Chip Childers" <ch...@gmail.com> wrote:
>> >>
>> >> >We are already (generally) AP for most infra changes really. I'd use
>> that
>> >> >model. Eventual consistency is better in this scenario.
>> >> >
>> >> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal
>> >> >><Ch...@citrix.com> wrote:
>> >> >>
>> >> >> I'd also like to highlight that it isn't a trivial problem.
>> >> >> Let's say there's 3 regions: this means there are 3 copies of the
>> user
>> >> >> database that are geographically separated by network links that
>> fail
>> >> >> quite often (orders of magnitude more than intra-DC networks).
>> >> >>
>> >> >> Here we run into the consequences of the CAP theorem [1].
>> >> >> We can either have a CP or AP system: either approach makes some
>> >> >>tradeoffs:
>> >> >> 1. If we run a AP system, then the challenge is to resolve
>> conflicting
>> >> >> updates
>> >> >> 2. If we run a CP system, then the challenge is to detect partitions
>> >> >> reliably and disallow updates during partitions.
>> >> >>
>> >> >> [1] http://en.wikipedia.org/wiki/CAP_theorem
>> >> >>
>> >> >>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org>
>> wrote:
>> >> >>>
>> >> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
>> >> >>> <Ch...@citrix.com> wrote:
>> >> >>>> It may be an admin burden, but it has to be optional. There are
>> other
>> >> >>>> ways
>> >> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).
>> >> >>>> A lot of service providers who run cloudstack have their own user
>> >> >>>> database
>> >> >>>> / portal. In their implementations the CloudStack database is not
>> the
>> >> >>>> master source of user records, but a slave.
>> >> >>>
>> >> >>> +1 to it being optional.
>> >> >>
>> >>
>> >>
>> >
>>
>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

All,

A little bit of updates after a long vacation,
I'm currently creating automated test scripts that randomly
create/delete/update domain/account/user objects in random regions to
trigger the sync-up and full scans regularly.
Once they are completed, I'll post it in the github also and submit the
review requests for this implementation.

Let me know if you have any comments.
Thanks
Alex Ough


On Wed, Dec 18, 2013 at 3:39 PM, Alex Ough <al...@sungard.com> wrote:

> All,
>
> I updated the wiki after some logic changes, so please review them,
> especially "Full Scan", which is newly introduced.
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
>
> And I implemented this functionality in Java and you can get the pull
> request of it here.
> (This does not include the 'full scan' yet and I'm currently working
> on this to finalize.)
> https://github.com/alexoughsg/Albatross/pull/1
>
> Especially, I really want to have your review on the "Full Scan" logic
> to confirm if it does not miss any cases.
> Thanks for your interest and your feedback will be very helpful.
> Alex Ough
>
> On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough <al...@sungard.com> wrote:
> > Good point, Chiradeep,
> >
> > I'm not sure if you reviewed my design doc in the wiki, but my design is
> to
> > just skip any actions for target resources that already took place by any
> > means.
> > But the issue is when conflict actions in the same resources (like
> create &
> > delete the same users) are enqueued in reversed orders, which is
> hopefully
> > rare.
> >
> > And to support consistency in the AP system, I'd like to provide a full
> sync
> > up, which will sync up all data in all region servers by selecting a
> region
> > as a master and force its data to other regions.
> >
> > Let me know what you think.
> > Thanks
> > Alex Ough
> >
> >
> > On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal
> > <Ch...@citrix.com> wrote:
> >>
> >> Missed this one. In a single region, the CloudStack DB is the master for
> >> most operations. If the infra is not in the state the DB says it should
> >> be, generally the approach is to whack it and make it conform. For some
> >> exceptions (live migration/related use cases are exceptions) the DB is
> the
> >> slave -- the point is that the inconsistency that inevitably arise in an
> >> AP system need a conflict resolution system. In a single region, the
> >> default is to assume that the MySQL DB is correct and handle other cases
> >> carefully.
> >>
> >> In a multi-region case, there is no master: disable an account in one
> >> region, and it may not propagate to the other regions for many
> hours/days.
> >> You could add a user in one region and then add the same user in a
> >> different region and conflict before the sync happens.
> >>
> >> This is of course not a problem unique to CloudStack -- people pay mucho
> >> dinero for Global AD/LDAP sync. I don't think this is a problem for
> >> CloudStack core to solve: I support the event-based mechanism for those
> >> who want this facility.
> >>
> >> Distributed systems are hard and research continues to try and make
> >> building these systems easier, but there are very few solutions for
> global
> >> state synchronization (Google Spanner comes to mind)
> >>
> >> --
> >> Chiradeep
> >>
> >>
> >> On 11/8/13 4:53 PM, "Chip Childers" <ch...@gmail.com> wrote:
> >>
> >> >We are already (generally) AP for most infra changes really. I'd use
> that
> >> >model. Eventual consistency is better in this scenario.
> >> >
> >> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal
> >> >><Ch...@citrix.com> wrote:
> >> >>
> >> >> I'd also like to highlight that it isn't a trivial problem.
> >> >> Let's say there's 3 regions: this means there are 3 copies of the
> user
> >> >> database that are geographically separated by network links that fail
> >> >> quite often (orders of magnitude more than intra-DC networks).
> >> >>
> >> >> Here we run into the consequences of the CAP theorem [1].
> >> >> We can either have a CP or AP system: either approach makes some
> >> >>tradeoffs:
> >> >> 1. If we run a AP system, then the challenge is to resolve
> conflicting
> >> >> updates
> >> >> 2. If we run a CP system, then the challenge is to detect partitions
> >> >> reliably and disallow updates during partitions.
> >> >>
> >> >> [1] http://en.wikipedia.org/wiki/CAP_theorem
> >> >>
> >> >>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org>
> wrote:
> >> >>>
> >> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
> >> >>> <Ch...@citrix.com> wrote:
> >> >>>> It may be an admin burden, but it has to be optional. There are
> other
> >> >>>> ways
> >> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).
> >> >>>> A lot of service providers who run cloudstack have their own user
> >> >>>> database
> >> >>>> / portal. In their implementations the CloudStack database is not
> the
> >> >>>> master source of user records, but a slave.
> >> >>>
> >> >>> +1 to it being optional.
> >> >>
> >>
> >>
> >
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

All,

I updated the wiki after some logic changes, so please review them,
especially "Full Scan", which is newly introduced.
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions

And I implemented this functionality in Java and you can get the pull
request of it here.
(This does not include the 'full scan' yet and I'm currently working
on this to finalize.)
https://github.com/alexoughsg/Albatross/pull/1

Especially, I really want to have your review on the "Full Scan" logic
to confirm if it does not miss any cases.
Thanks for your interest and your feedback will be very helpful.
Alex Ough

On Tue, Nov 12, 2013 at 6:00 PM, Alex Ough <al...@sungard.com> wrote:
> Good point, Chiradeep,
>
> I'm not sure if you reviewed my design doc in the wiki, but my design is to
> just skip any actions for target resources that already took place by any
> means.
> But the issue is when conflict actions in the same resources (like create &
> delete the same users) are enqueued in reversed orders, which is hopefully
> rare.
>
> And to support consistency in the AP system, I'd like to provide a full sync
> up, which will sync up all data in all region servers by selecting a region
> as a master and force its data to other regions.
>
> Let me know what you think.
> Thanks
> Alex Ough
>
>
> On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal
> <Ch...@citrix.com> wrote:
>>
>> Missed this one. In a single region, the CloudStack DB is the master for
>> most operations. If the infra is not in the state the DB says it should
>> be, generally the approach is to whack it and make it conform. For some
>> exceptions (live migration/related use cases are exceptions) the DB is the
>> slave -- the point is that the inconsistency that inevitably arise in an
>> AP system need a conflict resolution system. In a single region, the
>> default is to assume that the MySQL DB is correct and handle other cases
>> carefully.
>>
>> In a multi-region case, there is no master: disable an account in one
>> region, and it may not propagate to the other regions for many hours/days.
>> You could add a user in one region and then add the same user in a
>> different region and conflict before the sync happens.
>>
>> This is of course not a problem unique to CloudStack -- people pay mucho
>> dinero for Global AD/LDAP sync. I don't think this is a problem for
>> CloudStack core to solve: I support the event-based mechanism for those
>> who want this facility.
>>
>> Distributed systems are hard and research continues to try and make
>> building these systems easier, but there are very few solutions for global
>> state synchronization (Google Spanner comes to mind)
>>
>> --
>> Chiradeep
>>
>>
>> On 11/8/13 4:53 PM, "Chip Childers" <ch...@gmail.com> wrote:
>>
>> >We are already (generally) AP for most infra changes really. I'd use that
>> >model. Eventual consistency is better in this scenario.
>> >
>> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal
>> >><Ch...@citrix.com> wrote:
>> >>
>> >> I'd also like to highlight that it isn't a trivial problem.
>> >> Let's say there's 3 regions: this means there are 3 copies of the user
>> >> database that are geographically separated by network links that fail
>> >> quite often (orders of magnitude more than intra-DC networks).
>> >>
>> >> Here we run into the consequences of the CAP theorem [1].
>> >> We can either have a CP or AP system: either approach makes some
>> >>tradeoffs:
>> >> 1. If we run a AP system, then the challenge is to resolve conflicting
>> >> updates
>> >> 2. If we run a CP system, then the challenge is to detect partitions
>> >> reliably and disallow updates during partitions.
>> >>
>> >> [1] http://en.wikipedia.org/wiki/CAP_theorem
>> >>
>> >>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org> wrote:
>> >>>
>> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
>> >>> <Ch...@citrix.com> wrote:
>> >>>> It may be an admin burden, but it has to be optional. There are other
>> >>>> ways
>> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).
>> >>>> A lot of service providers who run cloudstack have their own user
>> >>>> database
>> >>>> / portal. In their implementations the CloudStack database is not the
>> >>>> master source of user records, but a slave.
>> >>>
>> >>> +1 to it being optional.
>> >>
>>
>>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

Good point, Chiradeep,

I'm not sure if you reviewed my design doc in the wiki, but my design is to
just skip any actions for target resources that already took place by any
means.
But the issue is when conflict actions in the same resources (like create &
delete the same users) are enqueued in reversed orders, which is hopefully
rare.

And to support consistency in the AP system, I'd like to provide a full
sync up, which will sync up all data in all region servers by selecting a
region as a master and force its data to other regions.

Let me know what you think.
Thanks
Alex Ough


On Tue, Nov 12, 2013 at 1:22 PM, Chiradeep Vittal <
Chiradeep.Vittal@citrix.com> wrote:

> Missed this one. In a single region, the CloudStack DB is the master for
> most operations. If the infra is not in the state the DB says it should
> be, generally the approach is to whack it and make it conform. For some
> exceptions (live migration/related use cases are exceptions) the DB is the
> slave -- the point is that the inconsistency that inevitably arise in an
> AP system need a conflict resolution system. In a single region, the
> default is to assume that the MySQL DB is correct and handle other cases
> carefully.
>
> In a multi-region case, there is no master: disable an account in one
> region, and it may not propagate to the other regions for many hours/days.
> You could add a user in one region and then add the same user in a
> different region and conflict before the sync happens.
>
> This is of course not a problem unique to CloudStack -- people pay mucho
> dinero for Global AD/LDAP sync. I don't think this is a problem for
> CloudStack core to solve: I support the event-based mechanism for those
> who want this facility.
>
> Distributed systems are hard and research continues to try and make
> building these systems easier, but there are very few solutions for global
> state synchronization (Google Spanner comes to mind)
>
> --
> Chiradeep
>
>
> On 11/8/13 4:53 PM, "Chip Childers" <ch...@gmail.com> wrote:
>
> >We are already (generally) AP for most infra changes really. I'd use that
> >model. Eventual consistency is better in this scenario.
> >
> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal
> >><Ch...@citrix.com> wrote:
> >>
> >> I'd also like to highlight that it isn't a trivial problem.
> >> Let's say there's 3 regions: this means there are 3 copies of the user
> >> database that are geographically separated by network links that fail
> >> quite often (orders of magnitude more than intra-DC networks).
> >>
> >> Here we run into the consequences of the CAP theorem [1].
> >> We can either have a CP or AP system: either approach makes some
> >>tradeoffs:
> >> 1. If we run a AP system, then the challenge is to resolve conflicting
> >> updates
> >> 2. If we run a CP system, then the challenge is to detect partitions
> >> reliably and disallow updates during partitions.
> >>
> >> [1] http://en.wikipedia.org/wiki/CAP_theorem
> >>
> >>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org> wrote:
> >>>
> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
> >>> <Ch...@citrix.com> wrote:
> >>>> It may be an admin burden, but it has to be optional. There are other
> >>>> ways
> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).
> >>>> A lot of service providers who run cloudstack have their own user
> >>>> database
> >>>> / portal. In their implementations the CloudStack database is not the
> >>>> master source of user records, but a slave.
> >>>
> >>> +1 to it being optional.
> >>
>
>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

All,

I'm new to the cloudstack project, so there can be something I missed for
consideration,
but the current region support seems to be inadequate (I hope this does not
offend anyone...)
because even if a customer can have multiple regions,
the resources in each region are totally separated like resources of
different customers.
So my personal opinion is that it looks like this functionality is
must-have not good-to-have to support the actual multiple regions.
Once again, I can be wrong, and please correct me if so.

Anyway, I've added more design specs in wiki including the consideration of
the support as a plug-in in resolving the #1 restriction.
Please review them and let me know what you think.

Thanks
Alex Ough

On Sat, Nov 9, 2013 at 8:20 AM, Daan Hoogland <da...@gmail.com>wrote:

> H Guys,
>
> Can you shoot at my claims below, please?
>
> syncing being optional does not conflict with the code being in the
> core server. It seems that making a plugin for this is misuse of the
> plugin mechanism. To me it is more of an option to switch on or of
> with a global setting, having some extra configuration.
>
> Also, cloudstack being a slave to the real user db is a separate issue
> from cloudstack syncing its copy.
>
> As for answerring the cap; this is a security answer as well as an
> operational answer.
>
> thanks,
> Daan
>
> On Sat, Nov 9, 2013 at 1:53 AM, Chip Childers <ch...@gmail.com>
> wrote:
> > We are already (generally) AP for most infra changes really. I'd use
> that model. Eventual consistency is better in this scenario.
> >
> >> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal <
> Chiradeep.Vittal@citrix.com> wrote:
> >>
> >> I'd also like to highlight that it isn't a trivial problem.
> >> Let's say there's 3 regions: this means there are 3 copies of the user
> >> database that are geographically separated by network links that fail
> >> quite often (orders of magnitude more than intra-DC networks).
> >>
> >> Here we run into the consequences of the CAP theorem [1].
> >> We can either have a CP or AP system: either approach makes some
> tradeoffs:
> >> 1. If we run a AP system, then the challenge is to resolve conflicting
> >> updates
> >> 2. If we run a CP system, then the challenge is to detect partitions
> >> reliably and disallow updates during partitions.
> >>
> >> [1] http://en.wikipedia.org/wiki/CAP_theorem
> >>
> >>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org> wrote:
> >>>
> >>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
> >>> <Ch...@citrix.com> wrote:
> >>>> It may be an admin burden, but it has to be optional. There are other
> >>>> ways
> >>>> to achieve global sync (e.g., LDAP/AD/Oauth).
> >>>> A lot of service providers who run cloudstack have their own user
> >>>> database
> >>>> / portal. In their implementations the CloudStack database is not the
> >>>> master source of user records, but a slave.
> >>>
> >>> +1 to it being optional.
> >>
>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Daan Hoogland <da...@gmail.com>]

H Guys,

Can you shoot at my claims below, please?

syncing being optional does not conflict with the code being in the
core server. It seems that making a plugin for this is misuse of the
plugin mechanism. To me it is more of an option to switch on or of
with a global setting, having some extra configuration.

Also, cloudstack being a slave to the real user db is a separate issue
from cloudstack syncing its copy.

As for answerring the cap; this is a security answer as well as an
operational answer.

thanks,
Daan

On Sat, Nov 9, 2013 at 1:53 AM, Chip Childers <ch...@gmail.com> wrote:
> We are already (generally) AP for most infra changes really. I'd use that model. Eventual consistency is better in this scenario.
>
>> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal <Ch...@citrix.com> wrote:
>>
>> I'd also like to highlight that it isn't a trivial problem.
>> Let's say there's 3 regions: this means there are 3 copies of the user
>> database that are geographically separated by network links that fail
>> quite often (orders of magnitude more than intra-DC networks).
>>
>> Here we run into the consequences of the CAP theorem [1].
>> We can either have a CP or AP system: either approach makes some tradeoffs:
>> 1. If we run a AP system, then the challenge is to resolve conflicting
>> updates
>> 2. If we run a CP system, then the challenge is to detect partitions
>> reliably and disallow updates during partitions.
>>
>> [1] http://en.wikipedia.org/wiki/CAP_theorem
>>
>>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org> wrote:
>>>
>>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
>>> <Ch...@citrix.com> wrote:
>>>> It may be an admin burden, but it has to be optional. There are other
>>>> ways
>>>> to achieve global sync (e.g., LDAP/AD/Oauth).
>>>> A lot of service providers who run cloudstack have their own user
>>>> database
>>>> / portal. In their implementations the CloudStack database is not the
>>>> master source of user records, but a slave.
>>>
>>> +1 to it being optional.
>>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Chiradeep Vittal <Ch...@citrix.com>]

Missed this one. In a single region, the CloudStack DB is the master for
most operations. If the infra is not in the state the DB says it should
be, generally the approach is to whack it and make it conform. For some
exceptions (live migration/related use cases are exceptions) the DB is the
slave -- the point is that the inconsistency that inevitably arise in an
AP system need a conflict resolution system. In a single region, the
default is to assume that the MySQL DB is correct and handle other cases
carefully.

In a multi-region case, there is no master: disable an account in one
region, and it may not propagate to the other regions for many hours/days.
You could add a user in one region and then add the same user in a
different region and conflict before the sync happens.
 
This is of course not a problem unique to CloudStack -- people pay mucho
dinero for Global AD/LDAP sync. I don't think this is a problem for
CloudStack core to solve: I support the event-based mechanism for those
who want this facility.

Distributed systems are hard and research continues to try and make
building these systems easier, but there are very few solutions for global
state synchronization (Google Spanner comes to mind)

--
Chiradeep


On 11/8/13 4:53 PM, "Chip Childers" <ch...@gmail.com> wrote:

>We are already (generally) AP for most infra changes really. I'd use that
>model. Eventual consistency is better in this scenario.
>
>> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal
>><Ch...@citrix.com> wrote:
>> 
>> I'd also like to highlight that it isn't a trivial problem.
>> Let's say there's 3 regions: this means there are 3 copies of the user
>> database that are geographically separated by network links that fail
>> quite often (orders of magnitude more than intra-DC networks).
>> 
>> Here we run into the consequences of the CAP theorem [1].
>> We can either have a CP or AP system: either approach makes some
>>tradeoffs:
>> 1. If we run a AP system, then the challenge is to resolve conflicting
>> updates
>> 2. If we run a CP system, then the challenge is to detect partitions
>> reliably and disallow updates during partitions.
>> 
>> [1] http://en.wikipedia.org/wiki/CAP_theorem
>> 
>>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org> wrote:
>>> 
>>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
>>> <Ch...@citrix.com> wrote:
>>>> It may be an admin burden, but it has to be optional. There are other
>>>> ways
>>>> to achieve global sync (e.g., LDAP/AD/Oauth).
>>>> A lot of service providers who run cloudstack have their own user
>>>> database
>>>> / portal. In their implementations the CloudStack database is not the
>>>> master source of user records, but a slave.
>>> 
>>> +1 to it being optional.
>> 

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Chip Childers <ch...@gmail.com>]

We are already (generally) AP for most infra changes really. I'd use that model. Eventual consistency is better in this scenario. 

> On Nov 8, 2013, at 6:49 PM, Chiradeep Vittal <Ch...@citrix.com> wrote:
> 
> I'd also like to highlight that it isn't a trivial problem.
> Let's say there's 3 regions: this means there are 3 copies of the user
> database that are geographically separated by network links that fail
> quite often (orders of magnitude more than intra-DC networks).
> 
> Here we run into the consequences of the CAP theorem [1].
> We can either have a CP or AP system: either approach makes some tradeoffs:
> 1. If we run a AP system, then the challenge is to resolve conflicting
> updates
> 2. If we run a CP system, then the challenge is to detect partitions
> reliably and disallow updates during partitions.
> 
> [1] http://en.wikipedia.org/wiki/CAP_theorem
> 
>> On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org> wrote:
>> 
>> On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
>> <Ch...@citrix.com> wrote:
>>> It may be an admin burden, but it has to be optional. There are other
>>> ways
>>> to achieve global sync (e.g., LDAP/AD/Oauth).
>>> A lot of service providers who run cloudstack have their own user
>>> database
>>> / portal. In their implementations the CloudStack database is not the
>>> master source of user records, but a slave.
>> 
>> +1 to it being optional.
> 

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Chiradeep Vittal <Ch...@citrix.com>]

I'd also like to highlight that it isn't a trivial problem.
Let's say there's 3 regions: this means there are 3 copies of the user
database that are geographically separated by network links that fail
quite often (orders of magnitude more than intra-DC networks).

Here we run into the consequences of the CAP theorem [1].
We can either have a CP or AP system: either approach makes some tradeoffs:
1. If we run a AP system, then the challenge is to resolve conflicting
updates
2. If we run a CP system, then the challenge is to detect partitions
reliably and disallow updates during partitions.

[1] http://en.wikipedia.org/wiki/CAP_theorem

On 11/7/13 11:58 AM, "Chip Childers" <ch...@apache.org> wrote:

>On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
><Ch...@citrix.com> wrote:
>> It may be an admin burden, but it has to be optional. There are other
>>ways
>> to achieve global sync (e.g., LDAP/AD/Oauth).
>> A lot of service providers who run cloudstack have their own user
>>database
>> / portal. In their implementations the CloudStack database is not the
>> master source of user records, but a slave.
>
>+1 to it being optional.

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Chip Childers <ch...@apache.org>]

On Thu, Nov 7, 2013 at 2:37 PM, Chiradeep Vittal
<Ch...@citrix.com> wrote:
> It may be an admin burden, but it has to be optional. There are other ways
> to achieve global sync (e.g., LDAP/AD/Oauth).
> A lot of service providers who run cloudstack have their own user database
> / portal. In their implementations the CloudStack database is not the
> master source of user records, but a slave.

+1 to it being optional.

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

I don't mind implementing it in the core server unless there is anyone who
thinks differently.
Anyone with another thought?

Thank Daan for your suggestion.
Alex Ough


On Thu, Nov 7, 2013 at 11:23 AM, Daan Hoogland <da...@gmail.com>wrote:

> Alex,
>
> Why would you want to make this a plugin? It sounds like a function of
> the core server. Don't you agree?
>
> regards,
> Daan
>
> On Wed, Nov 6, 2013 at 10:59 PM, Alex Ough <al...@sungard.com> wrote:
> > I'm having a difficulty finding documents about how to develop a plug-in.
> > Anyone to help me find one?
> >
> > Thanks in advance.
> > Alex Ough
> >
> >
> > On Tue, Nov 5, 2013 at 2:18 PM, Alex Ough <al...@sungard.com> wrote:
> >
> >> OK.
> >>
> >> 1) Do you mean the plug-in? If so, let me find out how to develop a
> >> plug-in and work on this.
> >> 2) Sure, let me add more information in the document.
> >>
> >> Thanks for your suggestions.
> >> Alex Ough
> >>
> >>
> >> On Tue, Nov 5, 2013 at 11:41 AM, Chip Childers <chipchilders@apache.org
> >wrote:
> >>
> >>> Alex,
> >>>
> >>> I've moved your page to the "Designs not committed to a release"
> >>> parent (instead of the 4.3 designs page), to align with both the Jira
> >>> record *and* the fact that feature freeze is about to happen for 4.3.
> >>>
> >>> As for the proposal itself, I have a couple of suggestions:
> >>>
> >>> 1) I'd like to see the implementation be part of the ACS runtime.
> >>> Having a separate python app for this sync feature seems like an admin
> >>> burden.
> >>>
> >>> 2) As far as the design document itself, I think that we need to see
> >>> more details on the proposed approach to sync, failure condition
> >>> handling, etc...
> >>>
> >>> -chip
> >>>
> >>>
> >>> On Mon, Nov 4, 2013 at 3:16 PM, Alex Ough <al...@sungard.com>
> wrote:
> >>> > All,
> >>> >
> >>> > Among the 2 approaches, I uploaded the implemented codes of the first
> >>> > approach, master-slave architecture, here.
> >>> > https://github.com/alexoughsg/albatross
> >>> >
> >>> > And here is the design doc in the wiki.
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
> >>> >
> >>> > Please review them and let me know what you think if you're
> interested!
> >>> > Thanks
> >>> > Alex Ough
> >>> >
> >>> >
> >>> >
> >>> > On Thu, Oct 31, 2013 at 6:51 PM, Alex Ough <al...@sungard.com>
> >>> wrote:
> >>> >
> >>> >> Great! Thanks a lot, Daan.
> >>> >>
> >>> >>
> >>> >> On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland <
> >>> daan.hoogland@gmail.com>wrote:
> >>> >>
> >>> >>> you are added to jira, Alex
> >>> >>>
> >>> >>> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough <al...@sungard.com>
> >>> wrote:
> >>> >>> > Thanks Chip, and can you also give a permission in Jira so that I
> >>> can
> >>> >>> > assign myself in its jira?
> >>> >>> >
> >>> >>> > Alex Ough
> >>> >>> >
> >>> >>> >
> >>> >>> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers <
> >>> chipchilders@apache.org
> >>> >>> >wrote:
> >>> >>> >
> >>> >>> >> Permission added.
> >>> >>> >>
> >>> >>> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote:
> >>> >>> >> > And I'd like to write the design document in the wiki page,
> but I
> >>> >>> don't
> >>> >>> >> > seem to have a permission to create pages.
> >>> >>> >> > So can anyone give me the permission?
> >>> >>> >> >
> >>> >>> >> > My account in the wiki is alex.ough@sungard.com
> >>> >>> >> >
> >>> >>> >> > Thanks in advance.
> >>> >>> >> > Alex Ough
> >>> >>> >> >
> >>> >>> >> >
> >>> >>> >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <
> >>> alex.ough@sungard.com>
> >>> >>> >> wrote:
> >>> >>> >> >
> >>> >>> >> > > I created a jira for this feature.
> >>> >>> >> > >
> >>> >>> >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992
> >>> >>> >> > >
> >>> >>> >> > > But it doesn't allow for me to assign it to myself, so any
> >>> >>> permission
> >>> >>> >> do I
> >>> >>> >> > > need for this?
> >>> >>> >> > > If so, can anyone give me this permission?
> >>> >>> >> > >
> >>> >>> >> > > If there is anything missing, let me know.
> >>> >>> >> > > Thanks
> >>> >>> >> > > Alex Ough
> >>> >>> >> > >
> >>> >>> >> > >
> >>> >>> >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <
> >>> >>> >> Kishan.Kavala@citrix.com>wrote:
> >>> >>> >> > >
> >>> >>> >> > >> > -----Original Message-----
> >>> >>> >> > >> > From: Alex Ough [mailto:alex.ough@sungard.com]
> >>> >>> >> > >> > Sent: Thursday, 17 October 2013 11:25 PM
> >>> >>> >> > >> > To: dev@cloudstack.apache.org;
> user@cloudstack.apache.org
> >>> >>> >> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among
> >>> >>> Multiple
> >>> >>> >> > >> > Regions
> >>> >>> >> > >> >
> >>> >>> >> > >> > All,
> >>> >>> >> > >> >
> >>> >>> >> > >> > Currently, under the environment of cloudstack with
> multiple
> >>> >>> >> regions,
> >>> >>> >> > >> each
> >>> >>> >> > >> > region has its own management server running with a
> separate
> >>> >>> >> database.
> >>> >>> >> > >> So if
> >>> >>> >> > >> > we want to support multiple regions and provide one
> point of
> >>> >>> entry
> >>> >>> >> for a
> >>> >>> >> > >> > customer, we need to duplicate domain/account/user
> >>> information
> >>> >>> of
> >>> >>> >> that
> >>> >>> >> > >> > customer to all of the databases of regions the customer
> >>> >>> accesses,
> >>> >>> >> > >> which will
> >>> >>> >> > >> > cause data discrepancies when users update those data
> >>> >>> independently
> >>> >>> >> in
> >>> >>> >> > >> each
> >>> >>> >> > >> > management server.
> >>> >>> >> > >> >
> >>> >>> >> > >> > So I'd like to provide a way to sync up the data using
> the
> >>> >>> messaging
> >>> >>> >> > >> system
> >>> >>> >> > >> > introduced in 4.1.0. Using the events from each
> management
> >>> >>> server,
> >>> >>> >> > >> updates
> >>> >>> >> > >> > from each region can be propagated to the rest regions
> and
> >>> they
> >>> >>> can
> >>> >>> >> be
> >>> >>> >> > >> > executed accordingly.
> >>> >>> >> > >> >
> >>> >>> >> > >> > I hope you guys have a chance to think about this and
> give
> >>> some
> >>> >>> >> > >> feedbacks if
> >>> >>> >> > >> > interested.
> >>> >>> >> > >> > Thanks in advance.
> >>> >>> >> > >> > Alex Ough
> >>> >>> >> > >>
> >>> >>> >> > >> [KK] Alex, it was discussed sometime back. Related thread
> [1].
> >>> >>> Sync up
> >>> >>> >> > >> using messaging system is the right way to go.
> >>> >>> >> > >>
> >>> >>> >> > >>
> >>> >>> >> > >> [1]
> >>> >>> >> > >>
> >>> >>> >>
> >>> >>>
> >>>
> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
> >>> >>> >> > >>
> >>> >>> >> > >>
> >>> >>> >> > >
> >>> >>> >>
> >>> >>> >>
> >>> >>>
> >>> >>>
> >>> >>
> >>>
> >>>
> >>
>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Daan Hoogland <da...@gmail.com>]

Alex,

Why would you want to make this a plugin? It sounds like a function of
the core server. Don't you agree?

regards,
Daan

On Wed, Nov 6, 2013 at 10:59 PM, Alex Ough <al...@sungard.com> wrote:
> I'm having a difficulty finding documents about how to develop a plug-in.
> Anyone to help me find one?
>
> Thanks in advance.
> Alex Ough
>
>
> On Tue, Nov 5, 2013 at 2:18 PM, Alex Ough <al...@sungard.com> wrote:
>
>> OK.
>>
>> 1) Do you mean the plug-in? If so, let me find out how to develop a
>> plug-in and work on this.
>> 2) Sure, let me add more information in the document.
>>
>> Thanks for your suggestions.
>> Alex Ough
>>
>>
>> On Tue, Nov 5, 2013 at 11:41 AM, Chip Childers <ch...@apache.org>wrote:
>>
>>> Alex,
>>>
>>> I've moved your page to the "Designs not committed to a release"
>>> parent (instead of the 4.3 designs page), to align with both the Jira
>>> record *and* the fact that feature freeze is about to happen for 4.3.
>>>
>>> As for the proposal itself, I have a couple of suggestions:
>>>
>>> 1) I'd like to see the implementation be part of the ACS runtime.
>>> Having a separate python app for this sync feature seems like an admin
>>> burden.
>>>
>>> 2) As far as the design document itself, I think that we need to see
>>> more details on the proposed approach to sync, failure condition
>>> handling, etc...
>>>
>>> -chip
>>>
>>>
>>> On Mon, Nov 4, 2013 at 3:16 PM, Alex Ough <al...@sungard.com> wrote:
>>> > All,
>>> >
>>> > Among the 2 approaches, I uploaded the implemented codes of the first
>>> > approach, master-slave architecture, here.
>>> > https://github.com/alexoughsg/albatross
>>> >
>>> > And here is the design doc in the wiki.
>>> >
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
>>> >
>>> > Please review them and let me know what you think if you're interested!
>>> > Thanks
>>> > Alex Ough
>>> >
>>> >
>>> >
>>> > On Thu, Oct 31, 2013 at 6:51 PM, Alex Ough <al...@sungard.com>
>>> wrote:
>>> >
>>> >> Great! Thanks a lot, Daan.
>>> >>
>>> >>
>>> >> On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland <
>>> daan.hoogland@gmail.com>wrote:
>>> >>
>>> >>> you are added to jira, Alex
>>> >>>
>>> >>> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough <al...@sungard.com>
>>> wrote:
>>> >>> > Thanks Chip, and can you also give a permission in Jira so that I
>>> can
>>> >>> > assign myself in its jira?
>>> >>> >
>>> >>> > Alex Ough
>>> >>> >
>>> >>> >
>>> >>> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers <
>>> chipchilders@apache.org
>>> >>> >wrote:
>>> >>> >
>>> >>> >> Permission added.
>>> >>> >>
>>> >>> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote:
>>> >>> >> > And I'd like to write the design document in the wiki page, but I
>>> >>> don't
>>> >>> >> > seem to have a permission to create pages.
>>> >>> >> > So can anyone give me the permission?
>>> >>> >> >
>>> >>> >> > My account in the wiki is alex.ough@sungard.com
>>> >>> >> >
>>> >>> >> > Thanks in advance.
>>> >>> >> > Alex Ough
>>> >>> >> >
>>> >>> >> >
>>> >>> >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <
>>> alex.ough@sungard.com>
>>> >>> >> wrote:
>>> >>> >> >
>>> >>> >> > > I created a jira for this feature.
>>> >>> >> > >
>>> >>> >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992
>>> >>> >> > >
>>> >>> >> > > But it doesn't allow for me to assign it to myself, so any
>>> >>> permission
>>> >>> >> do I
>>> >>> >> > > need for this?
>>> >>> >> > > If so, can anyone give me this permission?
>>> >>> >> > >
>>> >>> >> > > If there is anything missing, let me know.
>>> >>> >> > > Thanks
>>> >>> >> > > Alex Ough
>>> >>> >> > >
>>> >>> >> > >
>>> >>> >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <
>>> >>> >> Kishan.Kavala@citrix.com>wrote:
>>> >>> >> > >
>>> >>> >> > >> > -----Original Message-----
>>> >>> >> > >> > From: Alex Ough [mailto:alex.ough@sungard.com]
>>> >>> >> > >> > Sent: Thursday, 17 October 2013 11:25 PM
>>> >>> >> > >> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
>>> >>> >> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among
>>> >>> Multiple
>>> >>> >> > >> > Regions
>>> >>> >> > >> >
>>> >>> >> > >> > All,
>>> >>> >> > >> >
>>> >>> >> > >> > Currently, under the environment of cloudstack with multiple
>>> >>> >> regions,
>>> >>> >> > >> each
>>> >>> >> > >> > region has its own management server running with a separate
>>> >>> >> database.
>>> >>> >> > >> So if
>>> >>> >> > >> > we want to support multiple regions and provide one point of
>>> >>> entry
>>> >>> >> for a
>>> >>> >> > >> > customer, we need to duplicate domain/account/user
>>> information
>>> >>> of
>>> >>> >> that
>>> >>> >> > >> > customer to all of the databases of regions the customer
>>> >>> accesses,
>>> >>> >> > >> which will
>>> >>> >> > >> > cause data discrepancies when users update those data
>>> >>> independently
>>> >>> >> in
>>> >>> >> > >> each
>>> >>> >> > >> > management server.
>>> >>> >> > >> >
>>> >>> >> > >> > So I'd like to provide a way to sync up the data using the
>>> >>> messaging
>>> >>> >> > >> system
>>> >>> >> > >> > introduced in 4.1.0. Using the events from each management
>>> >>> server,
>>> >>> >> > >> updates
>>> >>> >> > >> > from each region can be propagated to the rest regions and
>>> they
>>> >>> can
>>> >>> >> be
>>> >>> >> > >> > executed accordingly.
>>> >>> >> > >> >
>>> >>> >> > >> > I hope you guys have a chance to think about this and give
>>> some
>>> >>> >> > >> feedbacks if
>>> >>> >> > >> > interested.
>>> >>> >> > >> > Thanks in advance.
>>> >>> >> > >> > Alex Ough
>>> >>> >> > >>
>>> >>> >> > >> [KK] Alex, it was discussed sometime back. Related thread [1].
>>> >>> Sync up
>>> >>> >> > >> using messaging system is the right way to go.
>>> >>> >> > >>
>>> >>> >> > >>
>>> >>> >> > >> [1]
>>> >>> >> > >>
>>> >>> >>
>>> >>>
>>> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
>>> >>> >> > >>
>>> >>> >> > >>
>>> >>> >> > >
>>> >>> >>
>>> >>> >>
>>> >>>
>>> >>>
>>> >>
>>>
>>>
>>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

I'm having a difficulty finding documents about how to develop a plug-in.
Anyone to help me find one?

Thanks in advance.
Alex Ough


On Tue, Nov 5, 2013 at 2:18 PM, Alex Ough <al...@sungard.com> wrote:

> OK.
>
> 1) Do you mean the plug-in? If so, let me find out how to develop a
> plug-in and work on this.
> 2) Sure, let me add more information in the document.
>
> Thanks for your suggestions.
> Alex Ough
>
>
> On Tue, Nov 5, 2013 at 11:41 AM, Chip Childers <ch...@apache.org>wrote:
>
>> Alex,
>>
>> I've moved your page to the "Designs not committed to a release"
>> parent (instead of the 4.3 designs page), to align with both the Jira
>> record *and* the fact that feature freeze is about to happen for 4.3.
>>
>> As for the proposal itself, I have a couple of suggestions:
>>
>> 1) I'd like to see the implementation be part of the ACS runtime.
>> Having a separate python app for this sync feature seems like an admin
>> burden.
>>
>> 2) As far as the design document itself, I think that we need to see
>> more details on the proposed approach to sync, failure condition
>> handling, etc...
>>
>> -chip
>>
>>
>> On Mon, Nov 4, 2013 at 3:16 PM, Alex Ough <al...@sungard.com> wrote:
>> > All,
>> >
>> > Among the 2 approaches, I uploaded the implemented codes of the first
>> > approach, master-slave architecture, here.
>> > https://github.com/alexoughsg/albatross
>> >
>> > And here is the design doc in the wiki.
>> >
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
>> >
>> > Please review them and let me know what you think if you're interested!
>> > Thanks
>> > Alex Ough
>> >
>> >
>> >
>> > On Thu, Oct 31, 2013 at 6:51 PM, Alex Ough <al...@sungard.com>
>> wrote:
>> >
>> >> Great! Thanks a lot, Daan.
>> >>
>> >>
>> >> On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland <
>> daan.hoogland@gmail.com>wrote:
>> >>
>> >>> you are added to jira, Alex
>> >>>
>> >>> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough <al...@sungard.com>
>> wrote:
>> >>> > Thanks Chip, and can you also give a permission in Jira so that I
>> can
>> >>> > assign myself in its jira?
>> >>> >
>> >>> > Alex Ough
>> >>> >
>> >>> >
>> >>> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers <
>> chipchilders@apache.org
>> >>> >wrote:
>> >>> >
>> >>> >> Permission added.
>> >>> >>
>> >>> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote:
>> >>> >> > And I'd like to write the design document in the wiki page, but I
>> >>> don't
>> >>> >> > seem to have a permission to create pages.
>> >>> >> > So can anyone give me the permission?
>> >>> >> >
>> >>> >> > My account in the wiki is alex.ough@sungard.com
>> >>> >> >
>> >>> >> > Thanks in advance.
>> >>> >> > Alex Ough
>> >>> >> >
>> >>> >> >
>> >>> >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <
>> alex.ough@sungard.com>
>> >>> >> wrote:
>> >>> >> >
>> >>> >> > > I created a jira for this feature.
>> >>> >> > >
>> >>> >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992
>> >>> >> > >
>> >>> >> > > But it doesn't allow for me to assign it to myself, so any
>> >>> permission
>> >>> >> do I
>> >>> >> > > need for this?
>> >>> >> > > If so, can anyone give me this permission?
>> >>> >> > >
>> >>> >> > > If there is anything missing, let me know.
>> >>> >> > > Thanks
>> >>> >> > > Alex Ough
>> >>> >> > >
>> >>> >> > >
>> >>> >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <
>> >>> >> Kishan.Kavala@citrix.com>wrote:
>> >>> >> > >
>> >>> >> > >> > -----Original Message-----
>> >>> >> > >> > From: Alex Ough [mailto:alex.ough@sungard.com]
>> >>> >> > >> > Sent: Thursday, 17 October 2013 11:25 PM
>> >>> >> > >> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
>> >>> >> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among
>> >>> Multiple
>> >>> >> > >> > Regions
>> >>> >> > >> >
>> >>> >> > >> > All,
>> >>> >> > >> >
>> >>> >> > >> > Currently, under the environment of cloudstack with multiple
>> >>> >> regions,
>> >>> >> > >> each
>> >>> >> > >> > region has its own management server running with a separate
>> >>> >> database.
>> >>> >> > >> So if
>> >>> >> > >> > we want to support multiple regions and provide one point of
>> >>> entry
>> >>> >> for a
>> >>> >> > >> > customer, we need to duplicate domain/account/user
>> information
>> >>> of
>> >>> >> that
>> >>> >> > >> > customer to all of the databases of regions the customer
>> >>> accesses,
>> >>> >> > >> which will
>> >>> >> > >> > cause data discrepancies when users update those data
>> >>> independently
>> >>> >> in
>> >>> >> > >> each
>> >>> >> > >> > management server.
>> >>> >> > >> >
>> >>> >> > >> > So I'd like to provide a way to sync up the data using the
>> >>> messaging
>> >>> >> > >> system
>> >>> >> > >> > introduced in 4.1.0. Using the events from each management
>> >>> server,
>> >>> >> > >> updates
>> >>> >> > >> > from each region can be propagated to the rest regions and
>> they
>> >>> can
>> >>> >> be
>> >>> >> > >> > executed accordingly.
>> >>> >> > >> >
>> >>> >> > >> > I hope you guys have a chance to think about this and give
>> some
>> >>> >> > >> feedbacks if
>> >>> >> > >> > interested.
>> >>> >> > >> > Thanks in advance.
>> >>> >> > >> > Alex Ough
>> >>> >> > >>
>> >>> >> > >> [KK] Alex, it was discussed sometime back. Related thread [1].
>> >>> Sync up
>> >>> >> > >> using messaging system is the right way to go.
>> >>> >> > >>
>> >>> >> > >>
>> >>> >> > >> [1]
>> >>> >> > >>
>> >>> >>
>> >>>
>> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
>> >>> >> > >>
>> >>> >> > >>
>> >>> >> > >
>> >>> >>
>> >>> >>
>> >>>
>> >>>
>> >>
>>
>>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

OK.

1) Do you mean the plug-in? If so, let me find out how to develop a plug-in
and work on this.
2) Sure, let me add more information in the document.

Thanks for your suggestions.
Alex Ough


On Tue, Nov 5, 2013 at 11:41 AM, Chip Childers <ch...@apache.org>wrote:

> Alex,
>
> I've moved your page to the "Designs not committed to a release"
> parent (instead of the 4.3 designs page), to align with both the Jira
> record *and* the fact that feature freeze is about to happen for 4.3.
>
> As for the proposal itself, I have a couple of suggestions:
>
> 1) I'd like to see the implementation be part of the ACS runtime.
> Having a separate python app for this sync feature seems like an admin
> burden.
>
> 2) As far as the design document itself, I think that we need to see
> more details on the proposed approach to sync, failure condition
> handling, etc...
>
> -chip
>
>
> On Mon, Nov 4, 2013 at 3:16 PM, Alex Ough <al...@sungard.com> wrote:
> > All,
> >
> > Among the 2 approaches, I uploaded the implemented codes of the first
> > approach, master-slave architecture, here.
> > https://github.com/alexoughsg/albatross
> >
> > And here is the design doc in the wiki.
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
> >
> > Please review them and let me know what you think if you're interested!
> > Thanks
> > Alex Ough
> >
> >
> >
> > On Thu, Oct 31, 2013 at 6:51 PM, Alex Ough <al...@sungard.com>
> wrote:
> >
> >> Great! Thanks a lot, Daan.
> >>
> >>
> >> On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland <daan.hoogland@gmail.com
> >wrote:
> >>
> >>> you are added to jira, Alex
> >>>
> >>> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough <al...@sungard.com>
> wrote:
> >>> > Thanks Chip, and can you also give a permission in Jira so that I can
> >>> > assign myself in its jira?
> >>> >
> >>> > Alex Ough
> >>> >
> >>> >
> >>> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers <
> chipchilders@apache.org
> >>> >wrote:
> >>> >
> >>> >> Permission added.
> >>> >>
> >>> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote:
> >>> >> > And I'd like to write the design document in the wiki page, but I
> >>> don't
> >>> >> > seem to have a permission to create pages.
> >>> >> > So can anyone give me the permission?
> >>> >> >
> >>> >> > My account in the wiki is alex.ough@sungard.com
> >>> >> >
> >>> >> > Thanks in advance.
> >>> >> > Alex Ough
> >>> >> >
> >>> >> >
> >>> >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <alex.ough@sungard.com
> >
> >>> >> wrote:
> >>> >> >
> >>> >> > > I created a jira for this feature.
> >>> >> > >
> >>> >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992
> >>> >> > >
> >>> >> > > But it doesn't allow for me to assign it to myself, so any
> >>> permission
> >>> >> do I
> >>> >> > > need for this?
> >>> >> > > If so, can anyone give me this permission?
> >>> >> > >
> >>> >> > > If there is anything missing, let me know.
> >>> >> > > Thanks
> >>> >> > > Alex Ough
> >>> >> > >
> >>> >> > >
> >>> >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <
> >>> >> Kishan.Kavala@citrix.com>wrote:
> >>> >> > >
> >>> >> > >> > -----Original Message-----
> >>> >> > >> > From: Alex Ough [mailto:alex.ough@sungard.com]
> >>> >> > >> > Sent: Thursday, 17 October 2013 11:25 PM
> >>> >> > >> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
> >>> >> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among
> >>> Multiple
> >>> >> > >> > Regions
> >>> >> > >> >
> >>> >> > >> > All,
> >>> >> > >> >
> >>> >> > >> > Currently, under the environment of cloudstack with multiple
> >>> >> regions,
> >>> >> > >> each
> >>> >> > >> > region has its own management server running with a separate
> >>> >> database.
> >>> >> > >> So if
> >>> >> > >> > we want to support multiple regions and provide one point of
> >>> entry
> >>> >> for a
> >>> >> > >> > customer, we need to duplicate domain/account/user
> information
> >>> of
> >>> >> that
> >>> >> > >> > customer to all of the databases of regions the customer
> >>> accesses,
> >>> >> > >> which will
> >>> >> > >> > cause data discrepancies when users update those data
> >>> independently
> >>> >> in
> >>> >> > >> each
> >>> >> > >> > management server.
> >>> >> > >> >
> >>> >> > >> > So I'd like to provide a way to sync up the data using the
> >>> messaging
> >>> >> > >> system
> >>> >> > >> > introduced in 4.1.0. Using the events from each management
> >>> server,
> >>> >> > >> updates
> >>> >> > >> > from each region can be propagated to the rest regions and
> they
> >>> can
> >>> >> be
> >>> >> > >> > executed accordingly.
> >>> >> > >> >
> >>> >> > >> > I hope you guys have a chance to think about this and give
> some
> >>> >> > >> feedbacks if
> >>> >> > >> > interested.
> >>> >> > >> > Thanks in advance.
> >>> >> > >> > Alex Ough
> >>> >> > >>
> >>> >> > >> [KK] Alex, it was discussed sometime back. Related thread [1].
> >>> Sync up
> >>> >> > >> using messaging system is the right way to go.
> >>> >> > >>
> >>> >> > >>
> >>> >> > >> [1]
> >>> >> > >>
> >>> >>
> >>>
> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
> >>> >> > >>
> >>> >> > >>
> >>> >> > >
> >>> >>
> >>> >>
> >>>
> >>>
> >>
>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Chiradeep Vittal <Ch...@citrix.com>]

It may be an admin burden, but it has to be optional. There are other ways
to achieve global sync (e.g., LDAP/AD/Oauth).
A lot of service providers who run cloudstack have their own user database
/ portal. In their implementations the CloudStack database is not the
master source of user records, but a slave.

On 11/5/13 12:41 PM, "Chip Childers" <ch...@apache.org> wrote:

>Alex,
>
>I've moved your page to the "Designs not committed to a release"
>parent (instead of the 4.3 designs page), to align with both the Jira
>record *and* the fact that feature freeze is about to happen for 4.3.
>
>As for the proposal itself, I have a couple of suggestions:
>
>1) I'd like to see the implementation be part of the ACS runtime.
>Having a separate python app for this sync feature seems like an admin
>burden.
>
>2) As far as the design document itself, I think that we need to see
>more details on the proposed approach to sync, failure condition
>handling, etc...
>
>-chip
>
>
>On Mon, Nov 4, 2013 at 3:16 PM, Alex Ough <al...@sungard.com> wrote:
>> All,
>>
>> Among the 2 approaches, I uploaded the implemented codes of the first
>> approach, master-slave architecture, here.
>> https://github.com/alexoughsg/albatross
>>
>> And here is the design doc in the wiki.
>> 
>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-Use
>>r+Sync+Up+Among+Multiple+Regions
>>
>> Please review them and let me know what you think if you're interested!
>> Thanks
>> Alex Ough
>>
>>
>>
>> On Thu, Oct 31, 2013 at 6:51 PM, Alex Ough <al...@sungard.com>
>>wrote:
>>
>>> Great! Thanks a lot, Daan.
>>>
>>>
>>> On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland
>>><da...@gmail.com>wrote:
>>>
>>>> you are added to jira, Alex
>>>>
>>>> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough <al...@sungard.com>
>>>>wrote:
>>>> > Thanks Chip, and can you also give a permission in Jira so that I
>>>>can
>>>> > assign myself in its jira?
>>>> >
>>>> > Alex Ough
>>>> >
>>>> >
>>>> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers
>>>><chipchilders@apache.org
>>>> >wrote:
>>>> >
>>>> >> Permission added.
>>>> >>
>>>> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote:
>>>> >> > And I'd like to write the design document in the wiki page, but I
>>>> don't
>>>> >> > seem to have a permission to create pages.
>>>> >> > So can anyone give me the permission?
>>>> >> >
>>>> >> > My account in the wiki is alex.ough@sungard.com
>>>> >> >
>>>> >> > Thanks in advance.
>>>> >> > Alex Ough
>>>> >> >
>>>> >> >
>>>> >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough
>>>><al...@sungard.com>
>>>> >> wrote:
>>>> >> >
>>>> >> > > I created a jira for this feature.
>>>> >> > >
>>>> >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992
>>>> >> > >
>>>> >> > > But it doesn't allow for me to assign it to myself, so any
>>>> permission
>>>> >> do I
>>>> >> > > need for this?
>>>> >> > > If so, can anyone give me this permission?
>>>> >> > >
>>>> >> > > If there is anything missing, let me know.
>>>> >> > > Thanks
>>>> >> > > Alex Ough
>>>> >> > >
>>>> >> > >
>>>> >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <
>>>> >> Kishan.Kavala@citrix.com>wrote:
>>>> >> > >
>>>> >> > >> > -----Original Message-----
>>>> >> > >> > From: Alex Ough [mailto:alex.ough@sungard.com]
>>>> >> > >> > Sent: Thursday, 17 October 2013 11:25 PM
>>>> >> > >> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
>>>> >> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among
>>>> Multiple
>>>> >> > >> > Regions
>>>> >> > >> >
>>>> >> > >> > All,
>>>> >> > >> >
>>>> >> > >> > Currently, under the environment of cloudstack with multiple
>>>> >> regions,
>>>> >> > >> each
>>>> >> > >> > region has its own management server running with a separate
>>>> >> database.
>>>> >> > >> So if
>>>> >> > >> > we want to support multiple regions and provide one point of
>>>> entry
>>>> >> for a
>>>> >> > >> > customer, we need to duplicate domain/account/user
>>>>information
>>>> of
>>>> >> that
>>>> >> > >> > customer to all of the databases of regions the customer
>>>> accesses,
>>>> >> > >> which will
>>>> >> > >> > cause data discrepancies when users update those data
>>>> independently
>>>> >> in
>>>> >> > >> each
>>>> >> > >> > management server.
>>>> >> > >> >
>>>> >> > >> > So I'd like to provide a way to sync up the data using the
>>>> messaging
>>>> >> > >> system
>>>> >> > >> > introduced in 4.1.0. Using the events from each management
>>>> server,
>>>> >> > >> updates
>>>> >> > >> > from each region can be propagated to the rest regions and
>>>>they
>>>> can
>>>> >> be
>>>> >> > >> > executed accordingly.
>>>> >> > >> >
>>>> >> > >> > I hope you guys have a chance to think about this and give
>>>>some
>>>> >> > >> feedbacks if
>>>> >> > >> > interested.
>>>> >> > >> > Thanks in advance.
>>>> >> > >> > Alex Ough
>>>> >> > >>
>>>> >> > >> [KK] Alex, it was discussed sometime back. Related thread [1].
>>>> Sync up
>>>> >> > >> using messaging system is the right way to go.
>>>> >> > >>
>>>> >> > >>
>>>> >> > >> [1]
>>>> >> > >>
>>>> >>
>>>> 
>>>>http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg2019
>>>>3.html
>>>> >> > >>
>>>> >> > >>
>>>> >> > >
>>>> >>
>>>> >>
>>>>
>>>>
>>>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Chip Childers <ch...@apache.org>]

Alex,

I've moved your page to the "Designs not committed to a release"
parent (instead of the 4.3 designs page), to align with both the Jira
record *and* the fact that feature freeze is about to happen for 4.3.

As for the proposal itself, I have a couple of suggestions:

1) I'd like to see the implementation be part of the ACS runtime.
Having a separate python app for this sync feature seems like an admin
burden.

2) As far as the design document itself, I think that we need to see
more details on the proposed approach to sync, failure condition
handling, etc...

-chip


On Mon, Nov 4, 2013 at 3:16 PM, Alex Ough <al...@sungard.com> wrote:
> All,
>
> Among the 2 approaches, I uploaded the implemented codes of the first
> approach, master-slave architecture, here.
> https://github.com/alexoughsg/albatross
>
> And here is the design doc in the wiki.
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions
>
> Please review them and let me know what you think if you're interested!
> Thanks
> Alex Ough
>
>
>
> On Thu, Oct 31, 2013 at 6:51 PM, Alex Ough <al...@sungard.com> wrote:
>
>> Great! Thanks a lot, Daan.
>>
>>
>> On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland <da...@gmail.com>wrote:
>>
>>> you are added to jira, Alex
>>>
>>> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough <al...@sungard.com> wrote:
>>> > Thanks Chip, and can you also give a permission in Jira so that I can
>>> > assign myself in its jira?
>>> >
>>> > Alex Ough
>>> >
>>> >
>>> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers <chipchilders@apache.org
>>> >wrote:
>>> >
>>> >> Permission added.
>>> >>
>>> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote:
>>> >> > And I'd like to write the design document in the wiki page, but I
>>> don't
>>> >> > seem to have a permission to create pages.
>>> >> > So can anyone give me the permission?
>>> >> >
>>> >> > My account in the wiki is alex.ough@sungard.com
>>> >> >
>>> >> > Thanks in advance.
>>> >> > Alex Ough
>>> >> >
>>> >> >
>>> >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <al...@sungard.com>
>>> >> wrote:
>>> >> >
>>> >> > > I created a jira for this feature.
>>> >> > >
>>> >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992
>>> >> > >
>>> >> > > But it doesn't allow for me to assign it to myself, so any
>>> permission
>>> >> do I
>>> >> > > need for this?
>>> >> > > If so, can anyone give me this permission?
>>> >> > >
>>> >> > > If there is anything missing, let me know.
>>> >> > > Thanks
>>> >> > > Alex Ough
>>> >> > >
>>> >> > >
>>> >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <
>>> >> Kishan.Kavala@citrix.com>wrote:
>>> >> > >
>>> >> > >> > -----Original Message-----
>>> >> > >> > From: Alex Ough [mailto:alex.ough@sungard.com]
>>> >> > >> > Sent: Thursday, 17 October 2013 11:25 PM
>>> >> > >> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
>>> >> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among
>>> Multiple
>>> >> > >> > Regions
>>> >> > >> >
>>> >> > >> > All,
>>> >> > >> >
>>> >> > >> > Currently, under the environment of cloudstack with multiple
>>> >> regions,
>>> >> > >> each
>>> >> > >> > region has its own management server running with a separate
>>> >> database.
>>> >> > >> So if
>>> >> > >> > we want to support multiple regions and provide one point of
>>> entry
>>> >> for a
>>> >> > >> > customer, we need to duplicate domain/account/user information
>>> of
>>> >> that
>>> >> > >> > customer to all of the databases of regions the customer
>>> accesses,
>>> >> > >> which will
>>> >> > >> > cause data discrepancies when users update those data
>>> independently
>>> >> in
>>> >> > >> each
>>> >> > >> > management server.
>>> >> > >> >
>>> >> > >> > So I'd like to provide a way to sync up the data using the
>>> messaging
>>> >> > >> system
>>> >> > >> > introduced in 4.1.0. Using the events from each management
>>> server,
>>> >> > >> updates
>>> >> > >> > from each region can be propagated to the rest regions and they
>>> can
>>> >> be
>>> >> > >> > executed accordingly.
>>> >> > >> >
>>> >> > >> > I hope you guys have a chance to think about this and give some
>>> >> > >> feedbacks if
>>> >> > >> > interested.
>>> >> > >> > Thanks in advance.
>>> >> > >> > Alex Ough
>>> >> > >>
>>> >> > >> [KK] Alex, it was discussed sometime back. Related thread [1].
>>> Sync up
>>> >> > >> using messaging system is the right way to go.
>>> >> > >>
>>> >> > >>
>>> >> > >> [1]
>>> >> > >>
>>> >>
>>> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
>>> >> > >>
>>> >> > >>
>>> >> > >
>>> >>
>>> >>
>>>
>>>
>>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

All,

Among the 2 approaches, I uploaded the implemented codes of the first
approach, master-slave architecture, here.
https://github.com/alexoughsg/albatross

And here is the design doc in the wiki.
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Domain-Account-User+Sync+Up+Among+Multiple+Regions

Please review them and let me know what you think if you're interested!
Thanks
Alex Ough



On Thu, Oct 31, 2013 at 6:51 PM, Alex Ough <al...@sungard.com> wrote:

> Great! Thanks a lot, Daan.
>
>
> On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland <da...@gmail.com>wrote:
>
>> you are added to jira, Alex
>>
>> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough <al...@sungard.com> wrote:
>> > Thanks Chip, and can you also give a permission in Jira so that I can
>> > assign myself in its jira?
>> >
>> > Alex Ough
>> >
>> >
>> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers <chipchilders@apache.org
>> >wrote:
>> >
>> >> Permission added.
>> >>
>> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote:
>> >> > And I'd like to write the design document in the wiki page, but I
>> don't
>> >> > seem to have a permission to create pages.
>> >> > So can anyone give me the permission?
>> >> >
>> >> > My account in the wiki is alex.ough@sungard.com
>> >> >
>> >> > Thanks in advance.
>> >> > Alex Ough
>> >> >
>> >> >
>> >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <al...@sungard.com>
>> >> wrote:
>> >> >
>> >> > > I created a jira for this feature.
>> >> > >
>> >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992
>> >> > >
>> >> > > But it doesn't allow for me to assign it to myself, so any
>> permission
>> >> do I
>> >> > > need for this?
>> >> > > If so, can anyone give me this permission?
>> >> > >
>> >> > > If there is anything missing, let me know.
>> >> > > Thanks
>> >> > > Alex Ough
>> >> > >
>> >> > >
>> >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <
>> >> Kishan.Kavala@citrix.com>wrote:
>> >> > >
>> >> > >> > -----Original Message-----
>> >> > >> > From: Alex Ough [mailto:alex.ough@sungard.com]
>> >> > >> > Sent: Thursday, 17 October 2013 11:25 PM
>> >> > >> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
>> >> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among
>> Multiple
>> >> > >> > Regions
>> >> > >> >
>> >> > >> > All,
>> >> > >> >
>> >> > >> > Currently, under the environment of cloudstack with multiple
>> >> regions,
>> >> > >> each
>> >> > >> > region has its own management server running with a separate
>> >> database.
>> >> > >> So if
>> >> > >> > we want to support multiple regions and provide one point of
>> entry
>> >> for a
>> >> > >> > customer, we need to duplicate domain/account/user information
>> of
>> >> that
>> >> > >> > customer to all of the databases of regions the customer
>> accesses,
>> >> > >> which will
>> >> > >> > cause data discrepancies when users update those data
>> independently
>> >> in
>> >> > >> each
>> >> > >> > management server.
>> >> > >> >
>> >> > >> > So I'd like to provide a way to sync up the data using the
>> messaging
>> >> > >> system
>> >> > >> > introduced in 4.1.0. Using the events from each management
>> server,
>> >> > >> updates
>> >> > >> > from each region can be propagated to the rest regions and they
>> can
>> >> be
>> >> > >> > executed accordingly.
>> >> > >> >
>> >> > >> > I hope you guys have a chance to think about this and give some
>> >> > >> feedbacks if
>> >> > >> > interested.
>> >> > >> > Thanks in advance.
>> >> > >> > Alex Ough
>> >> > >>
>> >> > >> [KK] Alex, it was discussed sometime back. Related thread [1].
>> Sync up
>> >> > >> using messaging system is the right way to go.
>> >> > >>
>> >> > >>
>> >> > >> [1]
>> >> > >>
>> >>
>> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
>> >> > >>
>> >> > >>
>> >> > >
>> >>
>> >>
>>
>>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

Great! Thanks a lot, Daan.


On Thu, Oct 31, 2013 at 4:58 PM, Daan Hoogland <da...@gmail.com>wrote:

> you are added to jira, Alex
>
> On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough <al...@sungard.com> wrote:
> > Thanks Chip, and can you also give a permission in Jira so that I can
> > assign myself in its jira?
> >
> > Alex Ough
> >
> >
> > On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers <chipchilders@apache.org
> >wrote:
> >
> >> Permission added.
> >>
> >> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote:
> >> > And I'd like to write the design document in the wiki page, but I
> don't
> >> > seem to have a permission to create pages.
> >> > So can anyone give me the permission?
> >> >
> >> > My account in the wiki is alex.ough@sungard.com
> >> >
> >> > Thanks in advance.
> >> > Alex Ough
> >> >
> >> >
> >> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <al...@sungard.com>
> >> wrote:
> >> >
> >> > > I created a jira for this feature.
> >> > >
> >> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992
> >> > >
> >> > > But it doesn't allow for me to assign it to myself, so any
> permission
> >> do I
> >> > > need for this?
> >> > > If so, can anyone give me this permission?
> >> > >
> >> > > If there is anything missing, let me know.
> >> > > Thanks
> >> > > Alex Ough
> >> > >
> >> > >
> >> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <
> >> Kishan.Kavala@citrix.com>wrote:
> >> > >
> >> > >> > -----Original Message-----
> >> > >> > From: Alex Ough [mailto:alex.ough@sungard.com]
> >> > >> > Sent: Thursday, 17 October 2013 11:25 PM
> >> > >> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
> >> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among
> Multiple
> >> > >> > Regions
> >> > >> >
> >> > >> > All,
> >> > >> >
> >> > >> > Currently, under the environment of cloudstack with multiple
> >> regions,
> >> > >> each
> >> > >> > region has its own management server running with a separate
> >> database.
> >> > >> So if
> >> > >> > we want to support multiple regions and provide one point of
> entry
> >> for a
> >> > >> > customer, we need to duplicate domain/account/user information of
> >> that
> >> > >> > customer to all of the databases of regions the customer
> accesses,
> >> > >> which will
> >> > >> > cause data discrepancies when users update those data
> independently
> >> in
> >> > >> each
> >> > >> > management server.
> >> > >> >
> >> > >> > So I'd like to provide a way to sync up the data using the
> messaging
> >> > >> system
> >> > >> > introduced in 4.1.0. Using the events from each management
> server,
> >> > >> updates
> >> > >> > from each region can be propagated to the rest regions and they
> can
> >> be
> >> > >> > executed accordingly.
> >> > >> >
> >> > >> > I hope you guys have a chance to think about this and give some
> >> > >> feedbacks if
> >> > >> > interested.
> >> > >> > Thanks in advance.
> >> > >> > Alex Ough
> >> > >>
> >> > >> [KK] Alex, it was discussed sometime back. Related thread [1].
> Sync up
> >> > >> using messaging system is the right way to go.
> >> > >>
> >> > >>
> >> > >> [1]
> >> > >>
> >>
> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
> >> > >>
> >> > >>
> >> > >
> >>
> >>
>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Daan Hoogland <da...@gmail.com>]

you are added to jira, Alex

On Thu, Oct 31, 2013 at 8:31 PM, Alex Ough <al...@sungard.com> wrote:
> Thanks Chip, and can you also give a permission in Jira so that I can
> assign myself in its jira?
>
> Alex Ough
>
>
> On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers <ch...@apache.org>wrote:
>
>> Permission added.
>>
>> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote:
>> > And I'd like to write the design document in the wiki page, but I don't
>> > seem to have a permission to create pages.
>> > So can anyone give me the permission?
>> >
>> > My account in the wiki is alex.ough@sungard.com
>> >
>> > Thanks in advance.
>> > Alex Ough
>> >
>> >
>> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <al...@sungard.com>
>> wrote:
>> >
>> > > I created a jira for this feature.
>> > >
>> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992
>> > >
>> > > But it doesn't allow for me to assign it to myself, so any permission
>> do I
>> > > need for this?
>> > > If so, can anyone give me this permission?
>> > >
>> > > If there is anything missing, let me know.
>> > > Thanks
>> > > Alex Ough
>> > >
>> > >
>> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <
>> Kishan.Kavala@citrix.com>wrote:
>> > >
>> > >> > -----Original Message-----
>> > >> > From: Alex Ough [mailto:alex.ough@sungard.com]
>> > >> > Sent: Thursday, 17 October 2013 11:25 PM
>> > >> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
>> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple
>> > >> > Regions
>> > >> >
>> > >> > All,
>> > >> >
>> > >> > Currently, under the environment of cloudstack with multiple
>> regions,
>> > >> each
>> > >> > region has its own management server running with a separate
>> database.
>> > >> So if
>> > >> > we want to support multiple regions and provide one point of entry
>> for a
>> > >> > customer, we need to duplicate domain/account/user information of
>> that
>> > >> > customer to all of the databases of regions the customer accesses,
>> > >> which will
>> > >> > cause data discrepancies when users update those data independently
>> in
>> > >> each
>> > >> > management server.
>> > >> >
>> > >> > So I'd like to provide a way to sync up the data using the messaging
>> > >> system
>> > >> > introduced in 4.1.0. Using the events from each management server,
>> > >> updates
>> > >> > from each region can be propagated to the rest regions and they can
>> be
>> > >> > executed accordingly.
>> > >> >
>> > >> > I hope you guys have a chance to think about this and give some
>> > >> feedbacks if
>> > >> > interested.
>> > >> > Thanks in advance.
>> > >> > Alex Ough
>> > >>
>> > >> [KK] Alex, it was discussed sometime back. Related thread [1]. Sync up
>> > >> using messaging system is the right way to go.
>> > >>
>> > >>
>> > >> [1]
>> > >>
>> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
>> > >>
>> > >>
>> > >
>>
>>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

Thanks Chip, and can you also give a permission in Jira so that I can
assign myself in its jira?

Alex Ough


On Thu, Oct 31, 2013 at 2:00 PM, Chip Childers <ch...@apache.org>wrote:

> Permission added.
>
> On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote:
> > And I'd like to write the design document in the wiki page, but I don't
> > seem to have a permission to create pages.
> > So can anyone give me the permission?
> >
> > My account in the wiki is alex.ough@sungard.com
> >
> > Thanks in advance.
> > Alex Ough
> >
> >
> > On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <al...@sungard.com>
> wrote:
> >
> > > I created a jira for this feature.
> > >
> > > https://issues.apache.org/jira/browse/CLOUDSTACK-4992
> > >
> > > But it doesn't allow for me to assign it to myself, so any permission
> do I
> > > need for this?
> > > If so, can anyone give me this permission?
> > >
> > > If there is anything missing, let me know.
> > > Thanks
> > > Alex Ough
> > >
> > >
> > > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <
> Kishan.Kavala@citrix.com>wrote:
> > >
> > >> > -----Original Message-----
> > >> > From: Alex Ough [mailto:alex.ough@sungard.com]
> > >> > Sent: Thursday, 17 October 2013 11:25 PM
> > >> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
> > >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple
> > >> > Regions
> > >> >
> > >> > All,
> > >> >
> > >> > Currently, under the environment of cloudstack with multiple
> regions,
> > >> each
> > >> > region has its own management server running with a separate
> database.
> > >> So if
> > >> > we want to support multiple regions and provide one point of entry
> for a
> > >> > customer, we need to duplicate domain/account/user information of
> that
> > >> > customer to all of the databases of regions the customer accesses,
> > >> which will
> > >> > cause data discrepancies when users update those data independently
> in
> > >> each
> > >> > management server.
> > >> >
> > >> > So I'd like to provide a way to sync up the data using the messaging
> > >> system
> > >> > introduced in 4.1.0. Using the events from each management server,
> > >> updates
> > >> > from each region can be propagated to the rest regions and they can
> be
> > >> > executed accordingly.
> > >> >
> > >> > I hope you guys have a chance to think about this and give some
> > >> feedbacks if
> > >> > interested.
> > >> > Thanks in advance.
> > >> > Alex Ough
> > >>
> > >> [KK] Alex, it was discussed sometime back. Related thread [1]. Sync up
> > >> using messaging system is the right way to go.
> > >>
> > >>
> > >> [1]
> > >>
> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
> > >>
> > >>
> > >
>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Chip Childers <ch...@apache.org>]

Permission added.

On Wed, Oct 30, 2013 at 12:19:23PM -0500, Alex Ough wrote:
> And I'd like to write the design document in the wiki page, but I don't
> seem to have a permission to create pages.
> So can anyone give me the permission?
> 
> My account in the wiki is alex.ough@sungard.com
> 
> Thanks in advance.
> Alex Ough
> 
> 
> On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <al...@sungard.com> wrote:
> 
> > I created a jira for this feature.
> >
> > https://issues.apache.org/jira/browse/CLOUDSTACK-4992
> >
> > But it doesn't allow for me to assign it to myself, so any permission do I
> > need for this?
> > If so, can anyone give me this permission?
> >
> > If there is anything missing, let me know.
> > Thanks
> > Alex Ough
> >
> >
> > On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <Ki...@citrix.com>wrote:
> >
> >> > -----Original Message-----
> >> > From: Alex Ough [mailto:alex.ough@sungard.com]
> >> > Sent: Thursday, 17 October 2013 11:25 PM
> >> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
> >> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple
> >> > Regions
> >> >
> >> > All,
> >> >
> >> > Currently, under the environment of cloudstack with multiple regions,
> >> each
> >> > region has its own management server running with a separate database.
> >> So if
> >> > we want to support multiple regions and provide one point of entry for a
> >> > customer, we need to duplicate domain/account/user information of that
> >> > customer to all of the databases of regions the customer accesses,
> >> which will
> >> > cause data discrepancies when users update those data independently in
> >> each
> >> > management server.
> >> >
> >> > So I'd like to provide a way to sync up the data using the messaging
> >> system
> >> > introduced in 4.1.0. Using the events from each management server,
> >> updates
> >> > from each region can be propagated to the rest regions and they can be
> >> > executed accordingly.
> >> >
> >> > I hope you guys have a chance to think about this and give some
> >> feedbacks if
> >> > interested.
> >> > Thanks in advance.
> >> > Alex Ough
> >>
> >> [KK] Alex, it was discussed sometime back. Related thread [1]. Sync up
> >> using messaging system is the right way to go.
> >>
> >>
> >> [1]
> >> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
> >>
> >>
> >

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Chip Childers <ch...@apache.org>]

(dropping users@ - try not to cross post please)

On Thu, Oct 31, 2013 at 12:13:18PM -0500, Alex Ough wrote:
> All,
> 
> While I'm waiting for the permissions, I'd like to bring up a discussion on
> the overall architecture to support this feature.
> 
> There can be 2 different approaches as below.
> 1. master - slave architecture : the manual changes are allowed only in one
> master management server, and those in other servers are either prohibited
> or discarded.
> 2. multiple source architecture : all management servers allow manual
> changes and any change in any server will be propagated to the rest of
> servers.
> 
> I'm not sure if we can impose the restriction of #1.
> If not, #2 is a little more complicated, but the only issue is to
> distinguish the events from either manual jobs or triggered automatic
> processing, which I'm currently working on.
> 
> Your comments/recommendations will be very appreciated.
> Thanks
> Alex Ough

Marcus - you might want to track this discussion, since I know that you
were surprised that there wasn't a native sync between regions as part
of the initial region feature.

-chip

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

All,

While I'm waiting for the permissions, I'd like to bring up a discussion on
the overall architecture to support this feature.

There can be 2 different approaches as below.
1. master - slave architecture : the manual changes are allowed only in one
master management server, and those in other servers are either prohibited
or discarded.
2. multiple source architecture : all management servers allow manual
changes and any change in any server will be propagated to the rest of
servers.

I'm not sure if we can impose the restriction of #1.
If not, #2 is a little more complicated, but the only issue is to
distinguish the events from either manual jobs or triggered automatic
processing, which I'm currently working on.

Your comments/recommendations will be very appreciated.
Thanks
Alex Ough


On Wed, Oct 30, 2013 at 12:19 PM, Alex Ough <al...@sungard.com> wrote:

> And I'd like to write the design document in the wiki page, but I don't
> seem to have a permission to create pages.
> So can anyone give me the permission?
>
> My account in the wiki is alex.ough@sungard.com
>
> Thanks in advance.
> Alex Ough
>
>
> On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <al...@sungard.com> wrote:
>
>> I created a jira for this feature.
>>
>> https://issues.apache.org/jira/browse/CLOUDSTACK-4992
>>
>> But it doesn't allow for me to assign it to myself, so any permission do
>> I need for this?
>> If so, can anyone give me this permission?
>>
>> If there is anything missing, let me know.
>> Thanks
>> Alex Ough
>>
>>
>> On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <Ki...@citrix.com>wrote:
>>
>>> > -----Original Message-----
>>> > From: Alex Ough [mailto:alex.ough@sungard.com]
>>> > Sent: Thursday, 17 October 2013 11:25 PM
>>> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
>>> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple
>>> > Regions
>>> >
>>> > All,
>>> >
>>> > Currently, under the environment of cloudstack with multiple regions,
>>> each
>>> > region has its own management server running with a separate database.
>>> So if
>>> > we want to support multiple regions and provide one point of entry for
>>> a
>>> > customer, we need to duplicate domain/account/user information of that
>>> > customer to all of the databases of regions the customer accesses,
>>> which will
>>> > cause data discrepancies when users update those data independently in
>>> each
>>> > management server.
>>> >
>>> > So I'd like to provide a way to sync up the data using the messaging
>>> system
>>> > introduced in 4.1.0. Using the events from each management server,
>>> updates
>>> > from each region can be propagated to the rest regions and they can be
>>> > executed accordingly.
>>> >
>>> > I hope you guys have a chance to think about this and give some
>>> feedbacks if
>>> > interested.
>>> > Thanks in advance.
>>> > Alex Ough
>>>
>>> [KK] Alex, it was discussed sometime back. Related thread [1]. Sync up
>>> using messaging system is the right way to go.
>>>
>>>
>>> [1]
>>> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
>>>
>>>
>>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

And I'd like to write the design document in the wiki page, but I don't
seem to have a permission to create pages.
So can anyone give me the permission?

My account in the wiki is alex.ough@sungard.com

Thanks in advance.
Alex Ough


On Tue, Oct 29, 2013 at 3:38 PM, Alex Ough <al...@sungard.com> wrote:

> I created a jira for this feature.
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-4992
>
> But it doesn't allow for me to assign it to myself, so any permission do I
> need for this?
> If so, can anyone give me this permission?
>
> If there is anything missing, let me know.
> Thanks
> Alex Ough
>
>
> On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <Ki...@citrix.com>wrote:
>
>> > -----Original Message-----
>> > From: Alex Ough [mailto:alex.ough@sungard.com]
>> > Sent: Thursday, 17 October 2013 11:25 PM
>> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
>> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple
>> > Regions
>> >
>> > All,
>> >
>> > Currently, under the environment of cloudstack with multiple regions,
>> each
>> > region has its own management server running with a separate database.
>> So if
>> > we want to support multiple regions and provide one point of entry for a
>> > customer, we need to duplicate domain/account/user information of that
>> > customer to all of the databases of regions the customer accesses,
>> which will
>> > cause data discrepancies when users update those data independently in
>> each
>> > management server.
>> >
>> > So I'd like to provide a way to sync up the data using the messaging
>> system
>> > introduced in 4.1.0. Using the events from each management server,
>> updates
>> > from each region can be propagated to the rest regions and they can be
>> > executed accordingly.
>> >
>> > I hope you guys have a chance to think about this and give some
>> feedbacks if
>> > interested.
>> > Thanks in advance.
>> > Alex Ough
>>
>> [KK] Alex, it was discussed sometime back. Related thread [1]. Sync up
>> using messaging system is the right way to go.
>>
>>
>> [1]
>> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
>>
>>
>

Re: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Alex Ough <al...@sungard.com>]

I created a jira for this feature.

https://issues.apache.org/jira/browse/CLOUDSTACK-4992

But it doesn't allow for me to assign it to myself, so any permission do I
need for this?
If so, can anyone give me this permission?

If there is anything missing, let me know.
Thanks
Alex Ough


On Fri, Oct 18, 2013 at 9:30 AM, Kishan Kavala <Ki...@citrix.com>wrote:

> > -----Original Message-----
> > From: Alex Ough [mailto:alex.ough@sungard.com]
> > Sent: Thursday, 17 October 2013 11:25 PM
> > To: dev@cloudstack.apache.org; user@cloudstack.apache.org
> > Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple
> > Regions
> >
> > All,
> >
> > Currently, under the environment of cloudstack with multiple regions,
> each
> > region has its own management server running with a separate database.
> So if
> > we want to support multiple regions and provide one point of entry for a
> > customer, we need to duplicate domain/account/user information of that
> > customer to all of the databases of regions the customer accesses, which
> will
> > cause data discrepancies when users update those data independently in
> each
> > management server.
> >
> > So I'd like to provide a way to sync up the data using the messaging
> system
> > introduced in 4.1.0. Using the events from each management server,
> updates
> > from each region can be propagated to the rest regions and they can be
> > executed accordingly.
> >
> > I hope you guys have a chance to think about this and give some
> feedbacks if
> > interested.
> > Thanks in advance.
> > Alex Ough
>
> [KK] Alex, it was discussed sometime back. Related thread [1]. Sync up
> using messaging system is the right way to go.
>
>
> [1]
> http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html
>
>

RE: [DISCUSS] Domain/Account/User Sync Up Among Multiple Regions

[Kishan Kavala <Ki...@citrix.com>]

> -----Original Message-----
> From: Alex Ough [mailto:alex.ough@sungard.com]
> Sent: Thursday, 17 October 2013 11:25 PM
> To: dev@cloudstack.apache.org; user@cloudstack.apache.org
> Subject: Fwd: [DISCUSS] Domain/Account/User Sync Up Among Multiple
> Regions
> 
> All,
> 
> Currently, under the environment of cloudstack with multiple regions, each
> region has its own management server running with a separate database. So if
> we want to support multiple regions and provide one point of entry for a
> customer, we need to duplicate domain/account/user information of that
> customer to all of the databases of regions the customer accesses, which will
> cause data discrepancies when users update those data independently in each
> management server.
> 
> So I'd like to provide a way to sync up the data using the messaging system
> introduced in 4.1.0. Using the events from each management server, updates
> from each region can be propagated to the rest regions and they can be
> executed accordingly.
> 
> I hope you guys have a chance to think about this and give some feedbacks if
> interested.
> Thanks in advance.
> Alex Ough

[KK] Alex, it was discussed sometime back. Related thread [1]. Sync up using messaging system is the right way to go. 


[1] http://www.mail-archive.com/cloudstack-dev@incubator.apache.org/msg20193.html