[DISCUSS] Remove async APIs from storage framework

[Darren Shepherd <da...@gmail.com>]

I've been reading over the storage code and have come to the conclusion 
that the async aspects of the storage framework should be removed.

Whenever one introduces an async pattern you have to give a lot of 
consideration to its use, benefits, and impact.  Within the context of 
ACS and given the current state of its code, I do not think it will be 
possible to realize any benefits of the current callback approach. 
Since nothing else in ACS uses callbacks, all of the async methods are 
essentially wrapped in synchronous calls.  So nothing as it stands is 
actually async.

Besides the current implementation, you need to conciser how you would 
expect an implementation of the storage framework to use the callback. 
The problem with callbacks is that they assume some in memory state. 
This means if the process/server crashes that state is lost.  Many will 
say just serialize the callback to the DB, but that is very impractical.

Since ACS doesn't actually stand in the data path, an async pattern 
won't really even allow it to have better performance.  ACS is just 
waiting for some storage operation to happen.  ACS can easily spawn 1000 
threads and have them all wait.  If you were to get to this point, you'd 
find that downstream you'll most likely have issues as you have 1000 
create template operations so its killing your filer.  So you will 
throttle storage operations to a level that won't kill your 
infrastructure and that level is no where near the scalability limits of 
threads.

The callbacks pattern really complicates the code and I see no real 
benefit.  Instead of spending a lot of effort trying to make all of ACS 
async to make it beneficial, I'd say that effort should be spent on 
making ACS idempotent and crash-only.  The point being, there's more 
beneficial things we can do with our time.

Given that only solidfire implements the new framework (and ACS legacy 
too), I would assume its a simple things for Edison to just go and 
quickly change it non-async.

Darren

Re: [DISCUSS] Remove async APIs from storage framework

[Amit Das <am...@cloudbyte.com>]

+1  for   "*If we wish to do async stuff it should be based off of java 5
futures or guava's ListenableFutures.  Those APIs and libraries are far
more natural to java devs in the wild.   Or we can get inspiration from
scala and akka on how to do async well in java.  In short, if we have a
real use case for async, then I'd prefer we use an off the shelf library."*

Regards,
Amit
*CloudByte Inc.* <http://www.cloudbyte.com/>


On Fri, Sep 6, 2013 at 7:33 AM, Darren Shepherd <darren.s.shepherd@gmail.com
> wrote:

> Yeah, I understand the timing aspect.  If there's already a lot in
> progress, then I'd rather not step in front of that train.  So as long as
> we agree going forward we look to guava or something, then we can figure
> out how to get there.
>
> Regarding the storage framework, can Edison or Min chime in on how
> difficult/impactful it would be to move to synchronous?  I could also help
> with it too.  The longer we wait the more the impact if more storage
> drivers get contributed.
>
> Darren
>
> On Sep 5, 2013, at 4:49 PM, Kelven Yang <ke...@citrix.com> wrote:
>
> > To tell the truth, I don't like the use of cglib in this case either, it
> > is per-request of making callback input available to IDE's intelli-sense
> > editing and be friendly to automatic code-refactoring through IDE
> > (Eclipse) which we do all the time.
> >
> > I'm fine with us to go with other standard approaches, my concern is the
> > timing.
> >
> > Kelven
> >
> > On 9/5/13 3:25 PM, "Darren Shepherd" <da...@gmail.com>
> wrote:
> >
> >> Kelvin,
> >>
> >> I really have some concerns about the entire async framework that was
> put
> >> into place as part of the ipc framework.  I don't know what VM sync is
> so
> >> I can't really comment too much on that.
> >>
> >> Basically my concern with the async framework is that it's yet another
> >> custom ACS thing.  The use of cglib in the callbacks (or futures?) is to
> >> obscure and is not clear to a new person coming into ACS (for example,
> >> me).
> >>
> >> If we wish to do async stuff it should be based off of java 5 futures or
> >> guava's ListenableFutures.  Those APIs and libraries are far more
> natural
> >> to java devs in the wild.   Or we can get inspiration from scala and
> akka
> >> on how to do async well in java.  In short, if we have a real use case
> >> for async, then I'd prefer we use an off the shelf library.
> >>
> >> In general I will lean towards most code being synchronous.  To answer
> >> Chiradeeps comment on how do we do insane scale.  That just requires
> true
> >> horizontal scalability of the management stack (which I'm not too sure
> if
> >> that's been fully proven beyond 3 servers and 30k hypervisors, but
> that's
> >> something we can address) and a really big database and somebody who
> >> enjoys really large failure domains.
> >>
> >> Darren
> >>
> >> On Sep 5, 2013, at 2:51 PM, Kelven Yang <ke...@citrix.com> wrote:
> >>
> >>> New VM sync and job flow handling are moving towards event-driven model
> >>> which is async natural. Not sure about the details on how storage
> >>> framework is using async mechanism, but argument on callback to assume
> >>> memory state does not stand as strong as it might be, since arguments
> in
> >>> any regular call pass information in memory the same way.
> >>>
> >>> Excessive callbacks (async method) usage pattern does encourage people
> >>> to
> >>> program in a wrong direction of async programming, I would prefer to
> >>> substitute callback with event signaling and maintain/access
> >>> flow-context
> >>> through separated facility with well defined API, this can prevent
> >>> developers to pass too much information directly and create a
> >>> tight-coupling pattern.
> >>>
> >>> However, I'm a little concerning about yet another storage refactoring
> >>> right after its previous refactoring work is barely done.
> >>>
> >>> Kelven
> >>>
> >>> On 9/5/13 12:57 PM, "Chiradeep Vittal" <Ch...@citrix.com>
> >>> wrote:
> >>>
> >>>> +1 for removing complexity especially if the sync pattern is being
> >>>> used on
> >>>> top of the async pattern.
> >>>> I see this behavior in the AgentManager.send as well -- even thought
> >>>> the
> >>>> AM
> >>>> is capable of async, practically nobody uses it as such.
> >>>>
> >>>> But I guess the question will arise : what if I do want more than 10^n
> >>>> long-running storage jobs
> >>>> (cos my cloud is as successful as AWS :))
> >>>>
> >>>> On 9/4/13 5:03 PM, "Darren Shepherd" <da...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> I've been reading over the storage code and have come to the
> >>>>> conclusion
> >>>>> that the async aspects of the storage framework should be removed.
> >>>>>
> >>>>> Whenever one introduces an async pattern you have to give a lot of
> >>>>> consideration to its use, benefits, and impact.  Within the context
> of
> >>>>> ACS and given the current state of its code, I do not think it will
> be
> >>>>> possible to realize any benefits of the current callback approach.
> >>>>> Since nothing else in ACS uses callbacks, all of the async methods
> are
> >>>>> essentially wrapped in synchronous calls.  So nothing as it stands is
> >>>>> actually async.
> >>>>>
> >>>>> Besides the current implementation, you need to conciser how you
> would
> >>>>> expect an implementation of the storage framework to use the
> callback.
> >>>>> The problem with callbacks is that they assume some in memory state.
> >>>>> This means if the process/server crashes that state is lost.  Many
> >>>>> will
> >>>>> say just serialize the callback to the DB, but that is very
> >>>>> impractical.
> >>>>>
> >>>>> Since ACS doesn't actually stand in the data path, an async pattern
> >>>>> won't really even allow it to have better performance.  ACS is just
> >>>>> waiting for some storage operation to happen.  ACS can easily spawn
> >>>>> 1000
> >>>>> threads and have them all wait.  If you were to get to this point,
> >>>>> you'd
> >>>>> find that downstream you'll most likely have issues as you have 1000
> >>>>> create template operations so its killing your filer.  So you will
> >>>>> throttle storage operations to a level that won't kill your
> >>>>> infrastructure and that level is no where near the scalability limits
> >>>>> of
> >>>>> threads.
> >>>>>
> >>>>> The callbacks pattern really complicates the code and I see no real
> >>>>> benefit.  Instead of spending a lot of effort trying to make all of
> >>>>> ACS
> >>>>> async to make it beneficial, I'd say that effort should be spent on
> >>>>> making ACS idempotent and crash-only.  The point being, there's more
> >>>>> beneficial things we can do with our time.
> >>>>>
> >>>>> Given that only solidfire implements the new framework (and ACS
> legacy
> >>>>> too), I would assume its a simple things for Edison to just go and
> >>>>> quickly change it non-async.
> >>>>>
> >>>>> Darren
> >
>

Re: [DISCUSS] Remove async APIs from storage framework

[Darren Shepherd <da...@gmail.com>]

Yeah, I understand the timing aspect.  If there's already a lot in progress, then I'd rather not step in front of that train.  So as long as we agree going forward we look to guava or something, then we can figure out how to get there.  

Regarding the storage framework, can Edison or Min chime in on how difficult/impactful it would be to move to synchronous?  I could also help with it too.  The longer we wait the more the impact if more storage drivers get contributed.

Darren

On Sep 5, 2013, at 4:49 PM, Kelven Yang <ke...@citrix.com> wrote:

> To tell the truth, I don't like the use of cglib in this case either, it
> is per-request of making callback input available to IDE's intelli-sense
> editing and be friendly to automatic code-refactoring through IDE
> (Eclipse) which we do all the time.
> 
> I'm fine with us to go with other standard approaches, my concern is the
> timing.
> 
> Kelven
> 
> On 9/5/13 3:25 PM, "Darren Shepherd" <da...@gmail.com> wrote:
> 
>> Kelvin,
>> 
>> I really have some concerns about the entire async framework that was put
>> into place as part of the ipc framework.  I don't know what VM sync is so
>> I can't really comment too much on that.
>> 
>> Basically my concern with the async framework is that it's yet another
>> custom ACS thing.  The use of cglib in the callbacks (or futures?) is to
>> obscure and is not clear to a new person coming into ACS (for example,
>> me).  
>> 
>> If we wish to do async stuff it should be based off of java 5 futures or
>> guava's ListenableFutures.  Those APIs and libraries are far more natural
>> to java devs in the wild.   Or we can get inspiration from scala and akka
>> on how to do async well in java.  In short, if we have a real use case
>> for async, then I'd prefer we use an off the shelf library.
>> 
>> In general I will lean towards most code being synchronous.  To answer
>> Chiradeeps comment on how do we do insane scale.  That just requires true
>> horizontal scalability of the management stack (which I'm not too sure if
>> that's been fully proven beyond 3 servers and 30k hypervisors, but that's
>> something we can address) and a really big database and somebody who
>> enjoys really large failure domains.
>> 
>> Darren
>> 
>> On Sep 5, 2013, at 2:51 PM, Kelven Yang <ke...@citrix.com> wrote:
>> 
>>> New VM sync and job flow handling are moving towards event-driven model
>>> which is async natural. Not sure about the details on how storage
>>> framework is using async mechanism, but argument on callback to assume
>>> memory state does not stand as strong as it might be, since arguments in
>>> any regular call pass information in memory the same way.
>>> 
>>> Excessive callbacks (async method) usage pattern does encourage people
>>> to
>>> program in a wrong direction of async programming, I would prefer to
>>> substitute callback with event signaling and maintain/access
>>> flow-context
>>> through separated facility with well defined API, this can prevent
>>> developers to pass too much information directly and create a
>>> tight-coupling pattern.
>>> 
>>> However, I'm a little concerning about yet another storage refactoring
>>> right after its previous refactoring work is barely done.
>>> 
>>> Kelven 
>>> 
>>> On 9/5/13 12:57 PM, "Chiradeep Vittal" <Ch...@citrix.com>
>>> wrote:
>>> 
>>>> +1 for removing complexity especially if the sync pattern is being
>>>> used on
>>>> top of the async pattern.
>>>> I see this behavior in the AgentManager.send as well -- even thought
>>>> the
>>>> AM 
>>>> is capable of async, practically nobody uses it as such.
>>>> 
>>>> But I guess the question will arise : what if I do want more than 10^n
>>>> long-running storage jobs
>>>> (cos my cloud is as successful as AWS :))
>>>> 
>>>> On 9/4/13 5:03 PM, "Darren Shepherd" <da...@gmail.com>
>>>> wrote:
>>>> 
>>>>> I've been reading over the storage code and have come to the
>>>>> conclusion
>>>>> that the async aspects of the storage framework should be removed.
>>>>> 
>>>>> Whenever one introduces an async pattern you have to give a lot of
>>>>> consideration to its use, benefits, and impact.  Within the context of
>>>>> ACS and given the current state of its code, I do not think it will be
>>>>> possible to realize any benefits of the current callback approach.
>>>>> Since nothing else in ACS uses callbacks, all of the async methods are
>>>>> essentially wrapped in synchronous calls.  So nothing as it stands is
>>>>> actually async.
>>>>> 
>>>>> Besides the current implementation, you need to conciser how you would
>>>>> expect an implementation of the storage framework to use the callback.
>>>>> The problem with callbacks is that they assume some in memory state.
>>>>> This means if the process/server crashes that state is lost.  Many
>>>>> will
>>>>> say just serialize the callback to the DB, but that is very
>>>>> impractical.
>>>>> 
>>>>> Since ACS doesn't actually stand in the data path, an async pattern
>>>>> won't really even allow it to have better performance.  ACS is just
>>>>> waiting for some storage operation to happen.  ACS can easily spawn
>>>>> 1000
>>>>> threads and have them all wait.  If you were to get to this point,
>>>>> you'd
>>>>> find that downstream you'll most likely have issues as you have 1000
>>>>> create template operations so its killing your filer.  So you will
>>>>> throttle storage operations to a level that won't kill your
>>>>> infrastructure and that level is no where near the scalability limits
>>>>> of
>>>>> threads.
>>>>> 
>>>>> The callbacks pattern really complicates the code and I see no real
>>>>> benefit.  Instead of spending a lot of effort trying to make all of
>>>>> ACS
>>>>> async to make it beneficial, I'd say that effort should be spent on
>>>>> making ACS idempotent and crash-only.  The point being, there's more
>>>>> beneficial things we can do with our time.
>>>>> 
>>>>> Given that only solidfire implements the new framework (and ACS legacy
>>>>> too), I would assume its a simple things for Edison to just go and
>>>>> quickly change it non-async.
>>>>> 
>>>>> Darren
> 

Re: [DISCUSS] Remove async APIs from storage framework

[Kelven Yang <ke...@citrix.com>]

To tell the truth, I don't like the use of cglib in this case either, it
is per-request of making callback input available to IDE's intelli-sense
editing and be friendly to automatic code-refactoring through IDE
(Eclipse) which we do all the time.

I'm fine with us to go with other standard approaches, my concern is the
timing.

Kelven

On 9/5/13 3:25 PM, "Darren Shepherd" <da...@gmail.com> wrote:

>Kelvin,
>
>I really have some concerns about the entire async framework that was put
>into place as part of the ipc framework.  I don't know what VM sync is so
>I can't really comment too much on that.
>
>Basically my concern with the async framework is that it's yet another
>custom ACS thing.  The use of cglib in the callbacks (or futures?) is to
>obscure and is not clear to a new person coming into ACS (for example,
>me).  
>
>If we wish to do async stuff it should be based off of java 5 futures or
>guava's ListenableFutures.  Those APIs and libraries are far more natural
>to java devs in the wild.   Or we can get inspiration from scala and akka
>on how to do async well in java.  In short, if we have a real use case
>for async, then I'd prefer we use an off the shelf library.
>
>In general I will lean towards most code being synchronous.  To answer
>Chiradeeps comment on how do we do insane scale.  That just requires true
>horizontal scalability of the management stack (which I'm not too sure if
>that's been fully proven beyond 3 servers and 30k hypervisors, but that's
>something we can address) and a really big database and somebody who
>enjoys really large failure domains.
>
>Darren
>
>On Sep 5, 2013, at 2:51 PM, Kelven Yang <ke...@citrix.com> wrote:
>
>> New VM sync and job flow handling are moving towards event-driven model
>> which is async natural. Not sure about the details on how storage
>> framework is using async mechanism, but argument on callback to assume
>> memory state does not stand as strong as it might be, since arguments in
>> any regular call pass information in memory the same way.
>> 
>> Excessive callbacks (async method) usage pattern does encourage people
>>to
>> program in a wrong direction of async programming, I would prefer to
>> substitute callback with event signaling and maintain/access
>>flow-context
>> through separated facility with well defined API, this can prevent
>> developers to pass too much information directly and create a
>> tight-coupling pattern.
>> 
>> However, I'm a little concerning about yet another storage refactoring
>> right after its previous refactoring work is barely done.
>> 
>> Kelven 
>> 
>> On 9/5/13 12:57 PM, "Chiradeep Vittal" <Ch...@citrix.com>
>>wrote:
>> 
>>> +1 for removing complexity especially if the sync pattern is being
>>>used on
>>> top of the async pattern.
>>> I see this behavior in the AgentManager.send as well -- even thought
>>>the
>>> AM 
>>> is capable of async, practically nobody uses it as such.
>>> 
>>> But I guess the question will arise : what if I do want more than 10^n
>>> long-running storage jobs
>>> (cos my cloud is as successful as AWS :))
>>> 
>>> On 9/4/13 5:03 PM, "Darren Shepherd" <da...@gmail.com>
>>>wrote:
>>> 
>>>> I've been reading over the storage code and have come to the
>>>>conclusion
>>>> that the async aspects of the storage framework should be removed.
>>>> 
>>>> Whenever one introduces an async pattern you have to give a lot of
>>>> consideration to its use, benefits, and impact.  Within the context of
>>>> ACS and given the current state of its code, I do not think it will be
>>>> possible to realize any benefits of the current callback approach.
>>>> Since nothing else in ACS uses callbacks, all of the async methods are
>>>> essentially wrapped in synchronous calls.  So nothing as it stands is
>>>> actually async.
>>>> 
>>>> Besides the current implementation, you need to conciser how you would
>>>> expect an implementation of the storage framework to use the callback.
>>>> The problem with callbacks is that they assume some in memory state.
>>>> This means if the process/server crashes that state is lost.  Many
>>>>will
>>>> say just serialize the callback to the DB, but that is very
>>>>impractical.
>>>> 
>>>> Since ACS doesn't actually stand in the data path, an async pattern
>>>> won't really even allow it to have better performance.  ACS is just
>>>> waiting for some storage operation to happen.  ACS can easily spawn
>>>>1000
>>>> threads and have them all wait.  If you were to get to this point,
>>>>you'd
>>>> find that downstream you'll most likely have issues as you have 1000
>>>> create template operations so its killing your filer.  So you will
>>>> throttle storage operations to a level that won't kill your
>>>> infrastructure and that level is no where near the scalability limits
>>>>of
>>>> threads.
>>>> 
>>>> The callbacks pattern really complicates the code and I see no real
>>>> benefit.  Instead of spending a lot of effort trying to make all of
>>>>ACS
>>>> async to make it beneficial, I'd say that effort should be spent on
>>>> making ACS idempotent and crash-only.  The point being, there's more
>>>> beneficial things we can do with our time.
>>>> 
>>>> Given that only solidfire implements the new framework (and ACS legacy
>>>> too), I would assume its a simple things for Edison to just go and
>>>> quickly change it non-async.
>>>> 
>>>> Darren
>> 

Re: [DISCUSS] Remove async APIs from storage framework

[Darren Shepherd <da...@gmail.com>]

Kelvin,

I really have some concerns about the entire async framework that was put into place as part of the ipc framework.  I don't know what VM sync is so I can't really comment too much on that.

Basically my concern with the async framework is that it's yet another custom ACS thing.  The use of cglib in the callbacks (or futures?) is to obscure and is not clear to a new person coming into ACS (for example, me).  

If we wish to do async stuff it should be based off of java 5 futures or guava's ListenableFutures.  Those APIs and libraries are far more natural to java devs in the wild.   Or we can get inspiration from scala and akka on how to do async well in java.  In short, if we have a real use case for async, then I'd prefer we use an off the shelf library.

In general I will lean towards most code being synchronous.  To answer Chiradeeps comment on how do we do insane scale.  That just requires true horizontal scalability of the management stack (which I'm not too sure if that's been fully proven beyond 3 servers and 30k hypervisors, but that's something we can address) and a really big database and somebody who enjoys really large failure domains. 

Darren

On Sep 5, 2013, at 2:51 PM, Kelven Yang <ke...@citrix.com> wrote:

> New VM sync and job flow handling are moving towards event-driven model
> which is async natural. Not sure about the details on how storage
> framework is using async mechanism, but argument on callback to assume
> memory state does not stand as strong as it might be, since arguments in
> any regular call pass information in memory the same way.
> 
> Excessive callbacks (async method) usage pattern does encourage people to
> program in a wrong direction of async programming, I would prefer to
> substitute callback with event signaling and maintain/access flow-context
> through separated facility with well defined API, this can prevent
> developers to pass too much information directly and create a
> tight-coupling pattern.
> 
> However, I'm a little concerning about yet another storage refactoring
> right after its previous refactoring work is barely done.
> 
> Kelven 
> 
> On 9/5/13 12:57 PM, "Chiradeep Vittal" <Ch...@citrix.com> wrote:
> 
>> +1 for removing complexity especially if the sync pattern is being used on
>> top of the async pattern.
>> I see this behavior in the AgentManager.send as well -- even thought the
>> AM 
>> is capable of async, practically nobody uses it as such.
>> 
>> But I guess the question will arise : what if I do want more than 10^n
>> long-running storage jobs
>> (cos my cloud is as successful as AWS :))
>> 
>> On 9/4/13 5:03 PM, "Darren Shepherd" <da...@gmail.com> wrote:
>> 
>>> I've been reading over the storage code and have come to the conclusion
>>> that the async aspects of the storage framework should be removed.
>>> 
>>> Whenever one introduces an async pattern you have to give a lot of
>>> consideration to its use, benefits, and impact.  Within the context of
>>> ACS and given the current state of its code, I do not think it will be
>>> possible to realize any benefits of the current callback approach.
>>> Since nothing else in ACS uses callbacks, all of the async methods are
>>> essentially wrapped in synchronous calls.  So nothing as it stands is
>>> actually async.
>>> 
>>> Besides the current implementation, you need to conciser how you would
>>> expect an implementation of the storage framework to use the callback.
>>> The problem with callbacks is that they assume some in memory state.
>>> This means if the process/server crashes that state is lost.  Many will
>>> say just serialize the callback to the DB, but that is very impractical.
>>> 
>>> Since ACS doesn't actually stand in the data path, an async pattern
>>> won't really even allow it to have better performance.  ACS is just
>>> waiting for some storage operation to happen.  ACS can easily spawn 1000
>>> threads and have them all wait.  If you were to get to this point, you'd
>>> find that downstream you'll most likely have issues as you have 1000
>>> create template operations so its killing your filer.  So you will
>>> throttle storage operations to a level that won't kill your
>>> infrastructure and that level is no where near the scalability limits of
>>> threads.
>>> 
>>> The callbacks pattern really complicates the code and I see no real
>>> benefit.  Instead of spending a lot of effort trying to make all of ACS
>>> async to make it beneficial, I'd say that effort should be spent on
>>> making ACS idempotent and crash-only.  The point being, there's more
>>> beneficial things we can do with our time.
>>> 
>>> Given that only solidfire implements the new framework (and ACS legacy
>>> too), I would assume its a simple things for Edison to just go and
>>> quickly change it non-async.
>>> 
>>> Darren
> 

Re: [DISCUSS] Remove async APIs from storage framework

[Kelven Yang <ke...@citrix.com>]

New VM sync and job flow handling are moving towards event-driven model
which is async natural. Not sure about the details on how storage
framework is using async mechanism, but argument on callback to assume
memory state does not stand as strong as it might be, since arguments in
any regular call pass information in memory the same way.

Excessive callbacks (async method) usage pattern does encourage people to
program in a wrong direction of async programming, I would prefer to
substitute callback with event signaling and maintain/access flow-context
through separated facility with well defined API, this can prevent
developers to pass too much information directly and create a
tight-coupling pattern.

However, I'm a little concerning about yet another storage refactoring
right after its previous refactoring work is barely done.

Kelven 

On 9/5/13 12:57 PM, "Chiradeep Vittal" <Ch...@citrix.com> wrote:

>+1 for removing complexity especially if the sync pattern is being used on
>top of the async pattern.
>I see this behavior in the AgentManager.send as well -- even thought the
>AM 
>is capable of async, practically nobody uses it as such.
>
>But I guess the question will arise : what if I do want more than 10^n
>long-running storage jobs
>(cos my cloud is as successful as AWS :))
>
>On 9/4/13 5:03 PM, "Darren Shepherd" <da...@gmail.com> wrote:
>
>>I've been reading over the storage code and have come to the conclusion
>>that the async aspects of the storage framework should be removed.
>>
>>Whenever one introduces an async pattern you have to give a lot of
>>consideration to its use, benefits, and impact.  Within the context of
>>ACS and given the current state of its code, I do not think it will be
>>possible to realize any benefits of the current callback approach.
>>Since nothing else in ACS uses callbacks, all of the async methods are
>>essentially wrapped in synchronous calls.  So nothing as it stands is
>>actually async.
>>
>>Besides the current implementation, you need to conciser how you would
>>expect an implementation of the storage framework to use the callback.
>>The problem with callbacks is that they assume some in memory state.
>>This means if the process/server crashes that state is lost.  Many will
>>say just serialize the callback to the DB, but that is very impractical.
>>
>>Since ACS doesn't actually stand in the data path, an async pattern
>>won't really even allow it to have better performance.  ACS is just
>>waiting for some storage operation to happen.  ACS can easily spawn 1000
>>threads and have them all wait.  If you were to get to this point, you'd
>>find that downstream you'll most likely have issues as you have 1000
>>create template operations so its killing your filer.  So you will
>>throttle storage operations to a level that won't kill your
>>infrastructure and that level is no where near the scalability limits of
>>threads.
>>
>>The callbacks pattern really complicates the code and I see no real
>>benefit.  Instead of spending a lot of effort trying to make all of ACS
>>async to make it beneficial, I'd say that effort should be spent on
>>making ACS idempotent and crash-only.  The point being, there's more
>>beneficial things we can do with our time.
>>
>>Given that only solidfire implements the new framework (and ACS legacy
>>too), I would assume its a simple things for Edison to just go and
>>quickly change it non-async.
>>
>>Darren
>

Re: [DISCUSS] Remove async APIs from storage framework

[Chiradeep Vittal <Ch...@citrix.com>]

+1 for removing complexity especially if the sync pattern is being used on
top of the async pattern.
I see this behavior in the AgentManager.send as well -- even thought the
AM 
is capable of async, practically nobody uses it as such.

But I guess the question will arise : what if I do want more than 10^n
long-running storage jobs
(cos my cloud is as successful as AWS :))

On 9/4/13 5:03 PM, "Darren Shepherd" <da...@gmail.com> wrote:

>I've been reading over the storage code and have come to the conclusion
>that the async aspects of the storage framework should be removed.
>
>Whenever one introduces an async pattern you have to give a lot of
>consideration to its use, benefits, and impact.  Within the context of
>ACS and given the current state of its code, I do not think it will be
>possible to realize any benefits of the current callback approach.
>Since nothing else in ACS uses callbacks, all of the async methods are
>essentially wrapped in synchronous calls.  So nothing as it stands is
>actually async.
>
>Besides the current implementation, you need to conciser how you would
>expect an implementation of the storage framework to use the callback.
>The problem with callbacks is that they assume some in memory state.
>This means if the process/server crashes that state is lost.  Many will
>say just serialize the callback to the DB, but that is very impractical.
>
>Since ACS doesn't actually stand in the data path, an async pattern
>won't really even allow it to have better performance.  ACS is just
>waiting for some storage operation to happen.  ACS can easily spawn 1000
>threads and have them all wait.  If you were to get to this point, you'd
>find that downstream you'll most likely have issues as you have 1000
>create template operations so its killing your filer.  So you will
>throttle storage operations to a level that won't kill your
>infrastructure and that level is no where near the scalability limits of
>threads.
>
>The callbacks pattern really complicates the code and I see no real
>benefit.  Instead of spending a lot of effort trying to make all of ACS
>async to make it beneficial, I'd say that effort should be spent on
>making ACS idempotent and crash-only.  The point being, there's more
>beneficial things we can do with our time.
>
>Given that only solidfire implements the new framework (and ACS legacy
>too), I would assume its a simple things for Edison to just go and
>quickly change it non-async.
>
>Darren