You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Prasanna Santhanam <ts...@apache.org> on 2013/06/25 15:32:43 UTC
How does the plugin model work for storage providers?
Hi
I noticed that all the storage providers are plugged in via
applicationContext by default. How does one plugin a custom provider -
say for example CompanyXStorageProvider?
On looking at the SolidFire implementation I found the plugin doesn't
actually come into play when running in either OSS or nonOSS mode.
IOW, the plugin isn't injected in either of - componentContext.xml.in
/ nonossComponentContext.xml.in. Is the merge still in progress?
Since these are not 'Adapters' so I don't know how to plugin my own
storage provider into the contexts - oss/non-oss. Unlike network
elements the DataStoreProvider, DataStoreLifeCycle seem independant and
don't follow the plugin model. How does this work?
Thanks,
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: How does the plugin model work for storage providers?
Posted by Mike Tutkowski <mi...@solidfire.com>.
Yeah, you have good points.
Perhaps we can implement them in 4.3, if it's not possible today.
On Tue, Jun 25, 2013 at 10:36 AM, Prasanna Santhanam <ts...@apache.org> wrote:
> yeah that includes the dependant driver and lifecycle of the
> corresponding provider. So it's a mix of componentInjection and XML
> based configuration. It's just an aesthetic detail to group the
> components together like network plugins. But it's useful to breakout
> plugins as separate blocks altogether.
>
> And it feels like anything that is a 'plugin' shouldn't have its
> configuration in the applicationContext because that couples it with a
> very basic management server config w/o the plugins.
>
> On Tue, Jun 25, 2013 at 08:35:41AM -0600, Mike Tutkowski wrote:
> > What I did was follow Edison's default plug-in as a template and used the
> > inject method to link in my life cycle and driver instances.
> >
> > @Override
> >
> > public boolean configure(Map<String, Object> params) {
> >
> > lifecycle =
> > ComponentContext.inject(SolidFirePrimaryDataStoreLifeCycle.class);
> >
> > driver = ComponentContext.inject(SolidfirePrimaryDataStoreDriver.
> > class);
> >
> > listener = ComponentContext.inject(new HypervisorHostListener() {
> >
> > public boolean hostConnect(long hostId, long poolId) {
> >
> > return true;
> >
> > }
> >
> >
> > public boolean hostDisconnected(long hostId, long poolId) {
> >
> > return true;
> >
> > }
> >
> > });
> >
> >
> > return true;
> >
> > }
> >
> >
> > On Tue, Jun 25, 2013 at 8:31 AM, Mike Tutkowski <
> > mike.tutkowski@solidfire.com> wrote:
> >
> > > Oh, I see...that's a good question.
> > >
> > > I'll have to defer that one to Edison.
> > >
> > >
> > > On Tue, Jun 25, 2013 at 8:28 AM, Prasanna Santhanam <tsp@apache.org
> >wrote:
> > >
> > >> Understood, I've done that. But I was wondering if there was a generic
> > >> way to group all components (driver, lifecycle, provider) of a
> > >> vendor-implementation into a logical spring context like say:
> > >>
> > >> In componentContext.xml.in
> > >>
> > >> <!-- Networking adapters -->
> > >> <bean id="ipDeployers"
> class="com.cloud.utils.component.AdapterList">
> > >> <property name="Adapters">
> > >> <list>
> > >> <ref bean="elasticLoadBalancerElement"/>
> > >> <ref bean="VirtualRouter"/>
> > >> <ref bean="VpcVirtualRouter"/>
> > >> <ref bean="NiciraNvp"/>
> > >> <ref bean="InternalLbVm"/>
> > >> </list>
> > >> </property>
> > >> </bean>
> > >>
> > >> On Tue, Jun 25, 2013 at 08:19:40AM -0600, Mike Tutkowski wrote:
> > >> > Hi,
> > >> >
> > >> > Yeah, John Burwell is finishing up the review process for the
> SolidFire
> > >> > plug-in, so - at present - the code is not in master.
> > >> >
> > >> > To try to answer your question, I had to modify the
> > >> > applicationContext.xml.in file.
> > >> >
> > >> > Here is the line I added:
> > >> >
> > >> > <bean id="solidFireDataStoreProvider"
> > >> >
> > >>
> class="org.apache.cloudstack.storage.datastore.provider.SolidfirePrimaryDataStoreProvider"
> > >> > />
> > >> >
> > >> > Talk to you later!
> > >> >
> > >> >
> > >> > On Tue, Jun 25, 2013 at 7:32 AM, Prasanna Santhanam <tsp@apache.org
> >
> > >> wrote:
> > >> >
> > >> > > Hi
> > >> > >
> > >> > > I noticed that all the storage providers are plugged in via
> > >> > > applicationContext by default. How does one plugin a custom
> provider -
> > >> > > say for example CompanyXStorageProvider?
> > >> > >
> > >> > > On looking at the SolidFire implementation I found the plugin
> doesn't
> > >> > > actually come into play when running in either OSS or nonOSS mode.
> > >> > > IOW, the plugin isn't injected in either of -
> componentContext.xml.in
> > >> > > / nonossComponentContext.xml.in. Is the merge still in progress?
> > >> > >
> > >> > > Since these are not 'Adapters' so I don't know how to plugin my
> own
> > >> > > storage provider into the contexts - oss/non-oss. Unlike network
> > >> > > elements the DataStoreProvider, DataStoreLifeCycle seem
> independant
> > >> and
> > >> > > don't follow the plugin model. How does this work?
> > >> > >
> > >> > > Thanks,
> > >> > >
> > >> > > --
> > >> > > Prasanna.,
> > >> > >
> > >> > > ------------------------
> > >> > > Powered by BigRock.com
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> > *Mike Tutkowski*
> > >> > *Senior CloudStack Developer, SolidFire Inc.*
> > >> > e: mike.tutkowski@solidfire.com
> > >> > o: 303.746.7302
> > >> > Advancing the way the world uses the
> > >> > cloud<http://solidfire.com/solution/overview/?video=play>
> > >> > *?*
> > >>
> > >> --
> > >> Prasanna.,
> > >>
> > >> ------------------------
> > >> Powered by BigRock.com
> > >>
> > >>
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the cloud<
> http://solidfire.com/solution/overview/?video=play>
> > > *?*
> > >
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *?*
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: How does the plugin model work for storage providers?
Posted by Prasanna Santhanam <ts...@apache.org>.
yeah that includes the dependant driver and lifecycle of the
corresponding provider. So it's a mix of componentInjection and XML
based configuration. It's just an aesthetic detail to group the
components together like network plugins. But it's useful to breakout
plugins as separate blocks altogether.
And it feels like anything that is a 'plugin' shouldn't have its
configuration in the applicationContext because that couples it with a
very basic management server config w/o the plugins.
On Tue, Jun 25, 2013 at 08:35:41AM -0600, Mike Tutkowski wrote:
> What I did was follow Edison's default plug-in as a template and used the
> inject method to link in my life cycle and driver instances.
>
> @Override
>
> public boolean configure(Map<String, Object> params) {
>
> lifecycle =
> ComponentContext.inject(SolidFirePrimaryDataStoreLifeCycle.class);
>
> driver = ComponentContext.inject(SolidfirePrimaryDataStoreDriver.
> class);
>
> listener = ComponentContext.inject(new HypervisorHostListener() {
>
> public boolean hostConnect(long hostId, long poolId) {
>
> return true;
>
> }
>
>
> public boolean hostDisconnected(long hostId, long poolId) {
>
> return true;
>
> }
>
> });
>
>
> return true;
>
> }
>
>
> On Tue, Jun 25, 2013 at 8:31 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
> > Oh, I see...that's a good question.
> >
> > I'll have to defer that one to Edison.
> >
> >
> > On Tue, Jun 25, 2013 at 8:28 AM, Prasanna Santhanam <ts...@apache.org>wrote:
> >
> >> Understood, I've done that. But I was wondering if there was a generic
> >> way to group all components (driver, lifecycle, provider) of a
> >> vendor-implementation into a logical spring context like say:
> >>
> >> In componentContext.xml.in
> >>
> >> <!-- Networking adapters -->
> >> <bean id="ipDeployers" class="com.cloud.utils.component.AdapterList">
> >> <property name="Adapters">
> >> <list>
> >> <ref bean="elasticLoadBalancerElement"/>
> >> <ref bean="VirtualRouter"/>
> >> <ref bean="VpcVirtualRouter"/>
> >> <ref bean="NiciraNvp"/>
> >> <ref bean="InternalLbVm"/>
> >> </list>
> >> </property>
> >> </bean>
> >>
> >> On Tue, Jun 25, 2013 at 08:19:40AM -0600, Mike Tutkowski wrote:
> >> > Hi,
> >> >
> >> > Yeah, John Burwell is finishing up the review process for the SolidFire
> >> > plug-in, so - at present - the code is not in master.
> >> >
> >> > To try to answer your question, I had to modify the
> >> > applicationContext.xml.in file.
> >> >
> >> > Here is the line I added:
> >> >
> >> > <bean id="solidFireDataStoreProvider"
> >> >
> >> class="org.apache.cloudstack.storage.datastore.provider.SolidfirePrimaryDataStoreProvider"
> >> > />
> >> >
> >> > Talk to you later!
> >> >
> >> >
> >> > On Tue, Jun 25, 2013 at 7:32 AM, Prasanna Santhanam <ts...@apache.org>
> >> wrote:
> >> >
> >> > > Hi
> >> > >
> >> > > I noticed that all the storage providers are plugged in via
> >> > > applicationContext by default. How does one plugin a custom provider -
> >> > > say for example CompanyXStorageProvider?
> >> > >
> >> > > On looking at the SolidFire implementation I found the plugin doesn't
> >> > > actually come into play when running in either OSS or nonOSS mode.
> >> > > IOW, the plugin isn't injected in either of - componentContext.xml.in
> >> > > / nonossComponentContext.xml.in. Is the merge still in progress?
> >> > >
> >> > > Since these are not 'Adapters' so I don't know how to plugin my own
> >> > > storage provider into the contexts - oss/non-oss. Unlike network
> >> > > elements the DataStoreProvider, DataStoreLifeCycle seem independant
> >> and
> >> > > don't follow the plugin model. How does this work?
> >> > >
> >> > > Thanks,
> >> > >
> >> > > --
> >> > > Prasanna.,
> >> > >
> >> > > ------------------------
> >> > > Powered by BigRock.com
> >> > >
> >> > >
> >> >
> >> >
> >> > --
> >> > *Mike Tutkowski*
> >> > *Senior CloudStack Developer, SolidFire Inc.*
> >> > e: mike.tutkowski@solidfire.com
> >> > o: 303.746.7302
> >> > Advancing the way the world uses the
> >> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > *?*
> >>
> >> --
> >> Prasanna.,
> >>
> >> ------------------------
> >> Powered by BigRock.com
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> > *?*
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *?*
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: How does the plugin model work for storage providers?
Posted by Mike Tutkowski <mi...@solidfire.com>.
What I did was follow Edison's default plug-in as a template and used the
inject method to link in my life cycle and driver instances.
@Override
public boolean configure(Map<String, Object> params) {
lifecycle =
ComponentContext.inject(SolidFirePrimaryDataStoreLifeCycle.class);
driver = ComponentContext.inject(SolidfirePrimaryDataStoreDriver.
class);
listener = ComponentContext.inject(new HypervisorHostListener() {
public boolean hostConnect(long hostId, long poolId) {
return true;
}
public boolean hostDisconnected(long hostId, long poolId) {
return true;
}
});
return true;
}
On Tue, Jun 25, 2013 at 8:31 AM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Oh, I see...that's a good question.
>
> I'll have to defer that one to Edison.
>
>
> On Tue, Jun 25, 2013 at 8:28 AM, Prasanna Santhanam <ts...@apache.org>wrote:
>
>> Understood, I've done that. But I was wondering if there was a generic
>> way to group all components (driver, lifecycle, provider) of a
>> vendor-implementation into a logical spring context like say:
>>
>> In componentContext.xml.in
>>
>> <!-- Networking adapters -->
>> <bean id="ipDeployers" class="com.cloud.utils.component.AdapterList">
>> <property name="Adapters">
>> <list>
>> <ref bean="elasticLoadBalancerElement"/>
>> <ref bean="VirtualRouter"/>
>> <ref bean="VpcVirtualRouter"/>
>> <ref bean="NiciraNvp"/>
>> <ref bean="InternalLbVm"/>
>> </list>
>> </property>
>> </bean>
>>
>> On Tue, Jun 25, 2013 at 08:19:40AM -0600, Mike Tutkowski wrote:
>> > Hi,
>> >
>> > Yeah, John Burwell is finishing up the review process for the SolidFire
>> > plug-in, so - at present - the code is not in master.
>> >
>> > To try to answer your question, I had to modify the
>> > applicationContext.xml.in file.
>> >
>> > Here is the line I added:
>> >
>> > <bean id="solidFireDataStoreProvider"
>> >
>> class="org.apache.cloudstack.storage.datastore.provider.SolidfirePrimaryDataStoreProvider"
>> > />
>> >
>> > Talk to you later!
>> >
>> >
>> > On Tue, Jun 25, 2013 at 7:32 AM, Prasanna Santhanam <ts...@apache.org>
>> wrote:
>> >
>> > > Hi
>> > >
>> > > I noticed that all the storage providers are plugged in via
>> > > applicationContext by default. How does one plugin a custom provider -
>> > > say for example CompanyXStorageProvider?
>> > >
>> > > On looking at the SolidFire implementation I found the plugin doesn't
>> > > actually come into play when running in either OSS or nonOSS mode.
>> > > IOW, the plugin isn't injected in either of - componentContext.xml.in
>> > > / nonossComponentContext.xml.in. Is the merge still in progress?
>> > >
>> > > Since these are not 'Adapters' so I don't know how to plugin my own
>> > > storage provider into the contexts - oss/non-oss. Unlike network
>> > > elements the DataStoreProvider, DataStoreLifeCycle seem independant
>> and
>> > > don't follow the plugin model. How does this work?
>> > >
>> > > Thanks,
>> > >
>> > > --
>> > > Prasanna.,
>> > >
>> > > ------------------------
>> > > Powered by BigRock.com
>> > >
>> > >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *?*
>>
>> --
>> Prasanna.,
>>
>> ------------------------
>> Powered by BigRock.com
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: How does the plugin model work for storage providers?
Posted by Mike Tutkowski <mi...@solidfire.com>.
Oh, I see...that's a good question.
I'll have to defer that one to Edison.
On Tue, Jun 25, 2013 at 8:28 AM, Prasanna Santhanam <ts...@apache.org> wrote:
> Understood, I've done that. But I was wondering if there was a generic
> way to group all components (driver, lifecycle, provider) of a
> vendor-implementation into a logical spring context like say:
>
> In componentContext.xml.in
>
> <!-- Networking adapters -->
> <bean id="ipDeployers" class="com.cloud.utils.component.AdapterList">
> <property name="Adapters">
> <list>
> <ref bean="elasticLoadBalancerElement"/>
> <ref bean="VirtualRouter"/>
> <ref bean="VpcVirtualRouter"/>
> <ref bean="NiciraNvp"/>
> <ref bean="InternalLbVm"/>
> </list>
> </property>
> </bean>
>
> On Tue, Jun 25, 2013 at 08:19:40AM -0600, Mike Tutkowski wrote:
> > Hi,
> >
> > Yeah, John Burwell is finishing up the review process for the SolidFire
> > plug-in, so - at present - the code is not in master.
> >
> > To try to answer your question, I had to modify the
> > applicationContext.xml.in file.
> >
> > Here is the line I added:
> >
> > <bean id="solidFireDataStoreProvider"
> >
> class="org.apache.cloudstack.storage.datastore.provider.SolidfirePrimaryDataStoreProvider"
> > />
> >
> > Talk to you later!
> >
> >
> > On Tue, Jun 25, 2013 at 7:32 AM, Prasanna Santhanam <ts...@apache.org>
> wrote:
> >
> > > Hi
> > >
> > > I noticed that all the storage providers are plugged in via
> > > applicationContext by default. How does one plugin a custom provider -
> > > say for example CompanyXStorageProvider?
> > >
> > > On looking at the SolidFire implementation I found the plugin doesn't
> > > actually come into play when running in either OSS or nonOSS mode.
> > > IOW, the plugin isn't injected in either of - componentContext.xml.in
> > > / nonossComponentContext.xml.in. Is the merge still in progress?
> > >
> > > Since these are not 'Adapters' so I don't know how to plugin my own
> > > storage provider into the contexts - oss/non-oss. Unlike network
> > > elements the DataStoreProvider, DataStoreLifeCycle seem independant and
> > > don't follow the plugin model. How does this work?
> > >
> > > Thanks,
> > >
> > > --
> > > Prasanna.,
> > >
> > > ------------------------
> > > Powered by BigRock.com
> > >
> > >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *?*
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: [MERGE] Simulator Storage Fixes to master (was Re: How does the
plugin model work for storage providers?)
Posted by Prasanna Santhanam <ts...@apache.org>.
On Fri, Jun 28, 2013 at 12:28:17AM +0000, Edison Su wrote:
> Hi Prasanna, thanks for your great effort to get simulator work. I
> reviewed all your changes, looks good to me, while there is only one
> thing that I don't like: the implementation of
> SimulatorImageStoreDriverImpl, so I rewrote it with a simpler
> implementation. The commit number is
> a2ec1daf8422d0a1c789e331a3261b05a4501060, please double check.
Thanks, it's got real good use for testing. We need to start using the
simulator more and more ... and make improvements to it.
Thanks for the fix. I didn't fully comprehend the AsyncDispatcher so
that's why the code plugged into the DB like that. Now it looks
clearer.
--
Prasanna.,
------------------------
Powered by BigRock.com
RE: [MERGE] Simulator Storage Fixes to master (was Re: How does the
plugin model work for storage providers?)
Posted by Edison Su <Ed...@citrix.com>.
Hi Prasanna, thanks for your great effort to get simulator work. I reviewed all your changes, looks good to me, while there is only one thing that I don't like: the implementation of SimulatorImageStoreDriverImpl, so I rewrote it with a simpler implementation. The commit number is a2ec1daf8422d0a1c789e331a3261b05a4501060, please double check.
> -----Original Message-----
> From: Prasanna Santhanam [mailto:tsp@apache.org]
> Sent: Thursday, June 27, 2013 8:07 AM
> To: CloudStack Dev
> Cc: Edison Su; Min Chen
> Subject: [MERGE] Simulator Storage Fixes to master (was Re: How does the
> plugin model work for storage providers?)
>
> On Wed, Jun 26, 2013 at 05:18:14PM +0530, Prasanna Santhanam wrote:
> > Reason I needed to do this is because I need to get the simulator to
> > use a separate spring context and override the default
> > CloudstackImageStore bean. The simulator bean has to intercept the
> > template download calls to say everything is present.
> >
> > But looks like the dependency graph for this goes up to the
> > DataMotionService -> DataMotionStrategy -> DataStoreManager ->
> > DataStoreProviderManager -> DataStoreProvider.
> >
> > I will push the changes to a branch for review.
> >
>
> This is a MERGE request to include the new storage subsystem support for
> the simulator. I'm including this as a merge since it touches some storage
> code (minor setters here and there) and reorganizes the Spring beans in to
> different bean contexts. With this 4.2 can run simulator and we can help
> jclouds run their live tests against the simulator end points as described in [1].
> And Ian to setup his pipeline for the LDAP testing [2]. And of course to get
> checkin tests working again [3]
>
> I got spring contexts working by overriding the beans loaded by management
> server. I've separated the beans such that OSS hypervisor related beans go
> into componentContext.xml and the VmWare and Solidfire related beans go
> into nonossComponentContext.xml.
>
>
> The deployVM, registerTemplate, downloadVolume operations work fine on
> the simulator now along with the checkin tests. I will run some more tests on
> the branch later with devcloud and real hypervisors tomorrow and merge it
> with master if there are no objections.
>
> branch: simulatorStorageFixes
> rebased on top of current master , will re-rebase if there are any other
> changes/reviews. The review-chunks are in the following commits:
>
> a6bb56b10dd25b2b248644e0dbb2a0f394499cee Set all templates/volumes to
> Ready in the simulator
> 373dd2b8b9db50c4c5b63ed9ddf419f1296977c1 Don't report back resource
> state to ResourceManagerImpl
> 2488694ca8ed2ff5f54dce26e518be9c2e920aa1 Group storage subsystem
> components for spring
> 724b3f423016546116a1a0fb499fab9f90658523 DataStore - provider, lifecycle,
> driver implementations for simulator
>
> [1] http://cloudstack.markmail.org/thread/ume76fqka7fi5nfz
> [2] http://markmail.org/message/hpy6vrbmxrpcqywx
> [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-
> +Testing+with+Python#Marvin-TestingwithPython-
>
> Thanks,
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
[MERGE] Simulator Storage Fixes to master (was Re: How does the
plugin model work for storage providers?)
Posted by Prasanna Santhanam <ts...@apache.org>.
On Wed, Jun 26, 2013 at 05:18:14PM +0530, Prasanna Santhanam wrote:
> Reason I needed to do this is because I need to get the simulator to use a
> separate spring context and override the default CloudstackImageStore bean. The
> simulator bean has to intercept the template download calls to say everything
> is present.
>
> But looks like the dependency graph for this goes up to the
> DataMotionService -> DataMotionStrategy -> DataStoreManager ->
> DataStoreProviderManager -> DataStoreProvider.
>
> I will push the changes to a branch for review.
>
This is a MERGE request to include the new storage subsystem support for the
simulator. I'm including this as a merge since it touches some storage code
(minor setters here and there) and reorganizes the Spring beans in
to different bean contexts. With this 4.2 can run simulator and we can
help jclouds run their live tests against the simulator end points as
described in [1]. And Ian to setup his pipeline for the LDAP testing
[2]. And of course to get checkin tests working again [3]
I got spring contexts working by overriding the beans loaded by
management server. I've separated the beans such that OSS hypervisor
related beans go into componentContext.xml and the VmWare and
Solidfire related beans go into nonossComponentContext.xml.
The deployVM, registerTemplate, downloadVolume operations work fine on
the simulator now along with the checkin tests. I will run some more tests on
the branch later with devcloud and real hypervisors tomorrow and merge
it with master if there are no objections.
branch: simulatorStorageFixes
rebased on top of current master , will re-rebase if there are any
other changes/reviews. The review-chunks are in the following commits:
a6bb56b10dd25b2b248644e0dbb2a0f394499cee Set all templates/volumes to Ready in the simulator
373dd2b8b9db50c4c5b63ed9ddf419f1296977c1 Don't report back resource state to ResourceManagerImpl
2488694ca8ed2ff5f54dce26e518be9c2e920aa1 Group storage subsystem components for spring
724b3f423016546116a1a0fb499fab9f90658523 DataStore - provider, lifecycle, driver implementations for simulator
[1] http://cloudstack.markmail.org/thread/ume76fqka7fi5nfz
[2] http://markmail.org/message/hpy6vrbmxrpcqywx
[3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python#Marvin-TestingwithPython-
Thanks,
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: How does the plugin model work for storage providers?
Posted by Prasanna Santhanam <ts...@apache.org>.
On Wed, Jun 26, 2013 at 03:53:20PM +0530, Prasanna Santhanam wrote:
> On Tue, Jun 25, 2013 at 11:06:36PM +0000, Edison Su wrote:
> > I would say, the adapter thing is the legacy stuff, all the storage
> > plugins will be managed by DataStoreProviderManager. But as you
> > said, we can group plugins into one place by using spring's syntax,
> > such as:
>
> > <bean id="storageProviders"
> > class="org.apach.***.datastoreproviderManagerImpl">
> > <property name="providerList">
> > <list>
> > <ref local="providerOne"/>
> > <ref local="providerTwo"/>
> > </list>
> > </property>
> > </bean>
> >
>
> I tried something like this:
> In applicationContext beanfactory:
> <bean id="dataStoreProviderManager"
> class="org.apache.cloudstack.storage.datastore.provider.DataStoreProviderManagerImpl">
> <property name="primaryDataStoreProviderMgr">
> <ref bean="primaryDataStoreProviderMgr"/>
> </property>
> <property name="imageStoreProviderMgr">
> <ref bean="imageStoreProviderMgr"/>
> </property>
> <property name="providers">
> <ref bean="storageProviders"/>
> </property>
> </bean>
>
> Since I'm trying to separate storageProviders as plugins I tried putting them
> in the componentContext.xml so I didn't use a <ref local="">. But there's a problem
>
> DataStoreProvider
> | PrimaryDataStoreProvider
> | ImageDataStoreProvider
>
> All the above are interfaces, so I can't inject them using beans and group all
> vendor-specific PrimaryDataStoreProviders (Solidfire, Default etc) and the
> imageStores (S3, Swift, NFS).
>
> Shouldn't they become abstract classes with getter/setter for the List<?> providers
> to be injected into the ProviderManager?
>
Fixed this as follows:
applicationContext.xml.in:
<!--
Data Store Provider Manager
-->
<bean id="dataStoreManagerImpl" class="org.apache.cloudstack.storage.datastore.DataStoreManagerImpl">
<property name="primaryStoreMgr">
<ref bean="#{dataStoreProviderManager.primaryDataStoreProviderMgr}"/>
</property>
<property name="imageDataStoreMgr">
<ref bean="#{dataStoreProviderManager.imageStoreProviderMgr}"/>
</property>
</bean>
<bean id="dataStoreProviderManager"
class="org.apache.cloudstack.storage.datastore.provider.DataStoreProviderManagerImpl">
<property name="primaryDataStoreProviderMgr">
<bean id="primaryDataStoreProviderMgr" class="org.apache.cloudstack.storage.datastore.manager.PrimaryDataStoreProviderManagerImpl" />
</property>
<property name="imageStoreProviderMgr">
<bean id="imageStoreProviderMgr" class="org.apache.cloudstack.storage.image.manager.ImageStoreProviderManagerImpl" />
</property>
<property name="providers">
<ref bean="storageProviders"/>
</property>
</bean>
componentContext.xml.in:
<!--Storage Providers-->
<bean id="storageProviders">
<property name="#{providers}">
<list>
<bean id="CloudStackPrimaryDataStoreProviderImpl" class="org.apache.cloudstack.storage.datastore.provider.CloudStackPrimaryDataStoreProviderImpl"/>
<!--<bean id="SolidfirePrimaryDataStoreProvider" class="org.apache.cloudstack.storage.datastore.provider.SolidfirePrimaryDataStoreProvider"/>-->
<!--<bean id="SamplePrimaryDataStoreProviderImpl" class="org.apache.cloudstack.storage.datastore.provider.SamplePrimaryDatastoreProviderImpl"/>-->
<bean id="CloudStackImageStoreProviderImpl" class="org.apache.cloudstack.storage.datastore.provider.CloudStackImageStoreProviderImpl"/>
<bean id="S3ImageStoreProviderImpl" class="org.apache.cloudstack.storage.datastore.provider.S3ImageStoreProviderImpl"/>
<bean id="SwiftImageStoreProviderImpl" class="org.apache.cloudstack.storage.datastore.provider.SwiftImageStoreProviderImpl"/>
</list>
</property>
</bean>
Reason I needed to do this is because I need to get the simulator to use a
separate spring context and override the default CloudstackImageStore bean. The
simulator bean has to intercept the template download calls to say everything
is present.
But looks like the dependency graph for this goes up to the
DataMotionService -> DataMotionStrategy -> DataStoreManager ->
DataStoreProviderManager -> DataStoreProvider.
I will push the changes to a branch for review.
Thanks,
------------------------
Powered by BigRock.com
Re: How does the plugin model work for storage providers?
Posted by Prasanna Santhanam <ts...@apache.org>.
On Tue, Jun 25, 2013 at 11:06:36PM +0000, Edison Su wrote:
> I would say, the adapter thing is the legacy stuff, all the storage
> plugins will be managed by DataStoreProviderManager. But as you
> said, we can group plugins into one place by using spring's syntax,
> such as:
> <bean id="storageProviders"
> class="org.apach.***.datastoreproviderManagerImpl">
> <property name="providerList">
> <list>
> <ref local="providerOne"/>
> <ref local="providerTwo"/>
> </list>
> </property>
> </bean>
>
I tried something like this:
In applicationContext beanfactory:
<bean id="dataStoreProviderManager"
class="org.apache.cloudstack.storage.datastore.provider.DataStoreProviderManagerImpl">
<property name="primaryDataStoreProviderMgr">
<ref bean="primaryDataStoreProviderMgr"/>
</property>
<property name="imageStoreProviderMgr">
<ref bean="imageStoreProviderMgr"/>
</property>
<property name="providers">
<ref bean="storageProviders"/>
</property>
</bean>
Since I'm trying to separate storageProviders as plugins I tried putting them
in the componentContext.xml so I didn't use a <ref local="">. But there's a problem
DataStoreProvider
| PrimaryDataStoreProvider
| ImageDataStoreProvider
All the above are interfaces, so I can't inject them using beans and group all
vendor-specific PrimaryDataStoreProviders (Solidfire, Default etc) and the
imageStores (S3, Swift, NFS).
Shouldn't they become abstract classes with getter/setter for the List<?> providers
to be injected into the ProviderManager?
PS: I'm no spring expert, just RTFM-ing to figure out the best way.
--
Prasanna.,
------------------------
Powered by BigRock.com
RE: How does the plugin model work for storage providers?
Posted by Edison Su <Ed...@citrix.com>.
I would say, the adapter thing is the legacy stuff, all the storage plugins will be managed by DataStoreProviderManager. But as you said, we can group plugins into one place by using spring's syntax, such as:
<bean id="storageProviders"
class="org.apach.***.datastoreproviderManagerImpl">
<property name="providerList">
<list>
<ref local="providerOne"/>
<ref local="providerTwo"/>
</list>
</property>
</bean>
> -----Original Message-----
> From: Prasanna Santhanam [mailto:tsp@apache.org]
> Sent: Tuesday, June 25, 2013 7:29 AM
> To: dev@cloudstack.apache.org
> Cc: Edison Su; John Burwell
> Subject: Re: How does the plugin model work for storage providers?
>
> Understood, I've done that. But I was wondering if there was a generic way
> to group all components (driver, lifecycle, provider) of a vendor-
> implementation into a logical spring context like say:
>
> In componentContext.xml.in
>
> <!-- Networking adapters -->
> <bean id="ipDeployers" class="com.cloud.utils.component.AdapterList">
> <property name="Adapters">
> <list>
> <ref bean="elasticLoadBalancerElement"/>
> <ref bean="VirtualRouter"/>
> <ref bean="VpcVirtualRouter"/>
> <ref bean="NiciraNvp"/>
> <ref bean="InternalLbVm"/>
> </list>
> </property>
> </bean>
>
> On Tue, Jun 25, 2013 at 08:19:40AM -0600, Mike Tutkowski wrote:
> > Hi,
> >
> > Yeah, John Burwell is finishing up the review process for the
> > SolidFire plug-in, so - at present - the code is not in master.
> >
> > To try to answer your question, I had to modify the
> > applicationContext.xml.in file.
> >
> > Here is the line I added:
> >
> > <bean id="solidFireDataStoreProvider"
> >
> class="org.apache.cloudstack.storage.datastore.provider.SolidfirePrimaryDat
> aStoreProvider"
> > />
> >
> > Talk to you later!
> >
> >
> > On Tue, Jun 25, 2013 at 7:32 AM, Prasanna Santhanam <ts...@apache.org>
> wrote:
> >
> > > Hi
> > >
> > > I noticed that all the storage providers are plugged in via
> > > applicationContext by default. How does one plugin a custom provider
> > > - say for example CompanyXStorageProvider?
> > >
> > > On looking at the SolidFire implementation I found the plugin
> > > doesn't actually come into play when running in either OSS or nonOSS
> mode.
> > > IOW, the plugin isn't injected in either of -
> > > componentContext.xml.in / nonossComponentContext.xml.in. Is the
> merge still in progress?
> > >
> > > Since these are not 'Adapters' so I don't know how to plugin my own
> > > storage provider into the contexts - oss/non-oss. Unlike network
> > > elements the DataStoreProvider, DataStoreLifeCycle seem independant
> > > and don't follow the plugin model. How does this work?
> > >
> > > Thanks,
> > >
> > > --
> > > Prasanna.,
> > >
> > > ------------------------
> > > Powered by BigRock.com
> > >
> > >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *?*
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
Re: How does the plugin model work for storage providers?
Posted by Prasanna Santhanam <ts...@apache.org>.
Understood, I've done that. But I was wondering if there was a generic
way to group all components (driver, lifecycle, provider) of a
vendor-implementation into a logical spring context like say:
In componentContext.xml.in
<!-- Networking adapters -->
<bean id="ipDeployers" class="com.cloud.utils.component.AdapterList">
<property name="Adapters">
<list>
<ref bean="elasticLoadBalancerElement"/>
<ref bean="VirtualRouter"/>
<ref bean="VpcVirtualRouter"/>
<ref bean="NiciraNvp"/>
<ref bean="InternalLbVm"/>
</list>
</property>
</bean>
On Tue, Jun 25, 2013 at 08:19:40AM -0600, Mike Tutkowski wrote:
> Hi,
>
> Yeah, John Burwell is finishing up the review process for the SolidFire
> plug-in, so - at present - the code is not in master.
>
> To try to answer your question, I had to modify the
> applicationContext.xml.in file.
>
> Here is the line I added:
>
> <bean id="solidFireDataStoreProvider"
> class="org.apache.cloudstack.storage.datastore.provider.SolidfirePrimaryDataStoreProvider"
> />
>
> Talk to you later!
>
>
> On Tue, Jun 25, 2013 at 7:32 AM, Prasanna Santhanam <ts...@apache.org> wrote:
>
> > Hi
> >
> > I noticed that all the storage providers are plugged in via
> > applicationContext by default. How does one plugin a custom provider -
> > say for example CompanyXStorageProvider?
> >
> > On looking at the SolidFire implementation I found the plugin doesn't
> > actually come into play when running in either OSS or nonOSS mode.
> > IOW, the plugin isn't injected in either of - componentContext.xml.in
> > / nonossComponentContext.xml.in. Is the merge still in progress?
> >
> > Since these are not 'Adapters' so I don't know how to plugin my own
> > storage provider into the contexts - oss/non-oss. Unlike network
> > elements the DataStoreProvider, DataStoreLifeCycle seem independant and
> > don't follow the plugin model. How does this work?
> >
> > Thanks,
> >
> > --
> > Prasanna.,
> >
> > ------------------------
> > Powered by BigRock.com
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *?*
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: How does the plugin model work for storage providers?
Posted by Mike Tutkowski <mi...@solidfire.com>.
This should help:
http://buildacloud.org/cloud-computing-vids/video/latest/storage-plug-ins-by-mike-tutkowski.html
It's a video of a presentation I did at a CloudStack Meetup in Santa Clara
on April 30th, where I talked about developing a storage plug-in for CS.
On Tue, Jun 25, 2013 at 8:19 AM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Hi,
>
> Yeah, John Burwell is finishing up the review process for the SolidFire
> plug-in, so - at present - the code is not in master.
>
> To try to answer your question, I had to modify the
> applicationContext.xml.in file.
>
> Here is the line I added:
>
> <bean id="solidFireDataStoreProvider"
> class="org.apache.cloudstack.storage.datastore.provider.SolidfirePrimaryDataStoreProvider"
> />
>
> Talk to you later!
>
>
> On Tue, Jun 25, 2013 at 7:32 AM, Prasanna Santhanam <ts...@apache.org>wrote:
>
>> Hi
>>
>> I noticed that all the storage providers are plugged in via
>> applicationContext by default. How does one plugin a custom provider -
>> say for example CompanyXStorageProvider?
>>
>> On looking at the SolidFire implementation I found the plugin doesn't
>> actually come into play when running in either OSS or nonOSS mode.
>> IOW, the plugin isn't injected in either of - componentContext.xml.in
>> / nonossComponentContext.xml.in. Is the merge still in progress?
>>
>> Since these are not 'Adapters' so I don't know how to plugin my own
>> storage provider into the contexts - oss/non-oss. Unlike network
>> elements the DataStoreProvider, DataStoreLifeCycle seem independant and
>> don't follow the plugin model. How does this work?
>>
>> Thanks,
>>
>> --
>> Prasanna.,
>>
>> ------------------------
>> Powered by BigRock.com
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
Re: How does the plugin model work for storage providers?
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi,
Yeah, John Burwell is finishing up the review process for the SolidFire
plug-in, so - at present - the code is not in master.
To try to answer your question, I had to modify the
applicationContext.xml.in file.
Here is the line I added:
<bean id="solidFireDataStoreProvider"
class="org.apache.cloudstack.storage.datastore.provider.SolidfirePrimaryDataStoreProvider"
/>
Talk to you later!
On Tue, Jun 25, 2013 at 7:32 AM, Prasanna Santhanam <ts...@apache.org> wrote:
> Hi
>
> I noticed that all the storage providers are plugged in via
> applicationContext by default. How does one plugin a custom provider -
> say for example CompanyXStorageProvider?
>
> On looking at the SolidFire implementation I found the plugin doesn't
> actually come into play when running in either OSS or nonOSS mode.
> IOW, the plugin isn't injected in either of - componentContext.xml.in
> / nonossComponentContext.xml.in. Is the merge still in progress?
>
> Since these are not 'Adapters' so I don't know how to plugin my own
> storage provider into the contexts - oss/non-oss. Unlike network
> elements the DataStoreProvider, DataStoreLifeCycle seem independant and
> don't follow the plugin model. How does this work?
>
> Thanks,
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*