You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by "Meyer, Carsten" <ca...@zeiss.com> on 2018/08/16 10:38:15 UTC

Help with extending Blob Store "filesystem" and Spring/Guice integration

We are trying to extend the Blob Store "filesystem" in such a way that Container are tied to a filesystem location instead of being placed a subdir of the basedir.

Our first approach was based on symbolic links. Here, each container is a symblic link pointing from the basedir subdirectory to the intended target. The advantage of this approach is that it is transparent to jclouds. Unfortunately, it is rather tricky to administer in Windows OS (the unix approach of symbolic links is straight-fowrward). Since we are running under Windows as well as Unix, this approach is not feasible.

Our alternative right implements the handling of the container/filesystem relationship.

We extended "FilesystemStorageStrategyImpl" by overriding the method "buildPathStartingFromBaseDir" in such a way that that path is built with the right container integration. The methods "deleteContainer" and "createContainer" are not needed in our use case.

The implementation is provided by our own "BlobStoreContextModul" so that within our application we only need to change the Provider ID.

BlobStoreContext context = ContextBuilder.newBuilder("containerperfilesystem")
        .overrides(properties)
        .buildView(BlobStoreContext.class);

Unfortunately we do not have experience with Guice, our application is Spring-based, and we have two questions:

1) Can the binding that is defined in the BlobStoreContextModul be overriden with our own strategy implementation, e.g. through the ContextBuilder?
2) Since our application is spring-based we are looking for a clean way to inject our own Container-Filesystem-Configuration from our application into the Guice binding or have the Guice object accessible from our application. Is this possible?

Thanks and best regards,
Carsten Meyer

Re: Help with extending Blob Store "filesystem" and Spring/Guice integration

Posted by Ignasi Barrera <na...@apache.org>.
The beauty of Guice and jclouds is that you should be able to extend
it by providing custom implementations for many things. You just have
to pass a Guice module with the custom bindings then creating the
context.

You could have something like:

class CustomBindings extends AbstractModule {
   @Override
   public void configure() {
      // Your custom bindings here, to provide custom implementations
for jclouds interfaces
   }
}

BlobStoreContext context = ContextBuilder.newBuilder("filesystem")
   .overrides(properties)
   .modules(ImmutableSet.of(new CustomBindings(), <other modules>))
   .buildView(BlobStoreContext.class);

When it comes to getting stuff from Guice, you can request the classes
to the Guice injector. The jclouds Context base class provides utility
methods to access that. For example:

context.utils().injector().getInstance(RequestedObjectClass.class);


HTH!

I.


On Fri, 17 Aug 2018 at 11:04, Meyer, Carsten <ca...@zeiss.com> wrote:
>
> Hi Andrew,
>
> thanks for your fast response!
>
>
>     On Thu, Aug 16, 2018 at 10:38:15AM +0000, Meyer, Carsten wrote:
>     > We are trying to extend the Blob Store "filesystem" in such a way that Container are tied to a filesystem location instead of being placed a subdir of the basedir.
>     >
>     > Our first approach was based on symbolic links. Here, each container is a symblic link pointing from the basedir subdirectory to the intended target. The advantage of this approach is that it is transparent to jclouds. Unfortunately, it is rather tricky to administer in Windows OS (the unix approach of symbolic links is straight-fowrward). Since we are running under Windows as well as Unix, this approach is not feasible.
>
>     I have little background in Windows but does `mklink /d src dst` solve
>     this?
>
> We can create symbolic links in this way, that works fine. Our problem is that symbolic links in Windows have a complexity that is higher than in Unix systems. Windows requires certain rights foe users to be able to create symbolic links. There is also a difference between remote and local links. Since our application is running on premise at the customer, administration of this complexity is not feasible. Also, there are still JDK bugs concering symbolic links under Windows but we were able to work around them.
>
>     > Our alternative right implements the handling of the container/filesystem relationship.
>     >
>     > We extended "FilesystemStorageStrategyImpl" by overriding the method "buildPathStartingFromBaseDir" in such a way that that path is built with the right container integration. The methods "deleteContainer" and "createContainer" are not needed in our use case.
>
>     I wonder if creating a filesystem BlobStore per container could also
>     address your use case?  These instances have little overhead and may
>     give you the control you want.
>
> We created that blob store and it basically works fine.
>
>     > The implementation is provided by our own "BlobStoreContextModul" so that within our application we only need to change the Provider ID.
>     >
>     > BlobStoreContext context = ContextBuilder.newBuilder("containerperfilesystem")
>     >         .overrides(properties)
>     >         .buildView(BlobStoreContext.class);
>     >
>     > Unfortunately we do not have experience with Guice, our application is Spring-based, and we have two questions:
>     >
>     > 1) Can the binding that is defined in the BlobStoreContextModul be overriden with our own strategy implementation, e.g. through the ContextBuilder?
>
>     With the caveat that these are not public or stable interfaces, you
>     should be able to extend FilesystemBlobStoreContextModule with a
>     configure method which stubs in your strategy class.  Then override the
>     builder via:
>
>         ContextBuilder.newBuilder("filesystem")
>             .defaultModule(CustomStrategy.class)
>             .build();
>
> So every Java class can be inserted via ."defaultModule()" - or maybe even instances?
> Currently (2.1.0) I cannot see this method in the ContextBuilder.
> Or am I misunderstanding something?
>
>     > 2) Since our application is spring-based we are looking for a clean way to inject our own Container-Filesystem-Configuration from our application into the Guice binding or have the Guice object accessible from our application. Is this possible?
>
>     Sorry I cannot help with Spring issues.
>
> Sorry, but the question was misleading.
> Since our application is non-guice-based we are looking for a clean way to inject our own Container Configuration from our application into the Guice binding or have the Guice object accessible from our application. Is this possible?
>
> Thanks again and best regards,
> Carsten
>
>

Re: Help with extending Blob Store "filesystem" and Spring/Guice integration

Posted by "Meyer, Carsten" <ca...@zeiss.com>.
Hi Andrew,

thanks for your fast response!


    On Thu, Aug 16, 2018 at 10:38:15AM +0000, Meyer, Carsten wrote:
    > We are trying to extend the Blob Store "filesystem" in such a way that Container are tied to a filesystem location instead of being placed a subdir of the basedir.
    > 
    > Our first approach was based on symbolic links. Here, each container is a symblic link pointing from the basedir subdirectory to the intended target. The advantage of this approach is that it is transparent to jclouds. Unfortunately, it is rather tricky to administer in Windows OS (the unix approach of symbolic links is straight-fowrward). Since we are running under Windows as well as Unix, this approach is not feasible.
    
    I have little background in Windows but does `mklink /d src dst` solve
    this?

We can create symbolic links in this way, that works fine. Our problem is that symbolic links in Windows have a complexity that is higher than in Unix systems. Windows requires certain rights foe users to be able to create symbolic links. There is also a difference between remote and local links. Since our application is running on premise at the customer, administration of this complexity is not feasible. Also, there are still JDK bugs concering symbolic links under Windows but we were able to work around them.
    
    > Our alternative right implements the handling of the container/filesystem relationship.
    > 
    > We extended "FilesystemStorageStrategyImpl" by overriding the method "buildPathStartingFromBaseDir" in such a way that that path is built with the right container integration. The methods "deleteContainer" and "createContainer" are not needed in our use case.
    
    I wonder if creating a filesystem BlobStore per container could also
    address your use case?  These instances have little overhead and may
    give you the control you want.
    
We created that blob store and it basically works fine.

    > The implementation is provided by our own "BlobStoreContextModul" so that within our application we only need to change the Provider ID.
    > 
    > BlobStoreContext context = ContextBuilder.newBuilder("containerperfilesystem")
    >         .overrides(properties)
    >         .buildView(BlobStoreContext.class);
    > 
    > Unfortunately we do not have experience with Guice, our application is Spring-based, and we have two questions:
    > 
    > 1) Can the binding that is defined in the BlobStoreContextModul be overriden with our own strategy implementation, e.g. through the ContextBuilder?
    
    With the caveat that these are not public or stable interfaces, you
    should be able to extend FilesystemBlobStoreContextModule with a
    configure method which stubs in your strategy class.  Then override the
    builder via:
    
        ContextBuilder.newBuilder("filesystem")
            .defaultModule(CustomStrategy.class)
            .build();
    
So every Java class can be inserted via ."defaultModule()" - or maybe even instances?
Currently (2.1.0) I cannot see this method in the ContextBuilder.
Or am I misunderstanding something?

    > 2) Since our application is spring-based we are looking for a clean way to inject our own Container-Filesystem-Configuration from our application into the Guice binding or have the Guice object accessible from our application. Is this possible?
    
    Sorry I cannot help with Spring issues.
    
Sorry, but the question was misleading.
Since our application is non-guice-based we are looking for a clean way to inject our own Container Configuration from our application into the Guice binding or have the Guice object accessible from our application. Is this possible?

Thanks again and best regards,
Carsten

Re: Help with extending Blob Store "filesystem" and Spring/Guice integration

Posted by Andrew Gaul <ga...@apache.org>.
On Thu, Aug 16, 2018 at 10:38:15AM +0000, Meyer, Carsten wrote:
> We are trying to extend the Blob Store "filesystem" in such a way that Container are tied to a filesystem location instead of being placed a subdir of the basedir.
> 
> Our first approach was based on symbolic links. Here, each container is a symblic link pointing from the basedir subdirectory to the intended target. The advantage of this approach is that it is transparent to jclouds. Unfortunately, it is rather tricky to administer in Windows OS (the unix approach of symbolic links is straight-fowrward). Since we are running under Windows as well as Unix, this approach is not feasible.

I have little background in Windows but does `mklink /d src dst` solve
this?

> Our alternative right implements the handling of the container/filesystem relationship.
> 
> We extended "FilesystemStorageStrategyImpl" by overriding the method "buildPathStartingFromBaseDir" in such a way that that path is built with the right container integration. The methods "deleteContainer" and "createContainer" are not needed in our use case.

I wonder if creating a filesystem BlobStore per container could also
address your use case?  These instances have little overhead and may
give you the control you want.

> The implementation is provided by our own "BlobStoreContextModul" so that within our application we only need to change the Provider ID.
> 
> BlobStoreContext context = ContextBuilder.newBuilder("containerperfilesystem")
>         .overrides(properties)
>         .buildView(BlobStoreContext.class);
> 
> Unfortunately we do not have experience with Guice, our application is Spring-based, and we have two questions:
> 
> 1) Can the binding that is defined in the BlobStoreContextModul be overriden with our own strategy implementation, e.g. through the ContextBuilder?

With the caveat that these are not public or stable interfaces, you
should be able to extend FilesystemBlobStoreContextModule with a
configure method which stubs in your strategy class.  Then override the
builder via:

    ContextBuilder.newBuilder("filesystem")
        .defaultModule(CustomStrategy.class)
        .build();

> 2) Since our application is spring-based we are looking for a clean way to inject our own Container-Filesystem-Configuration from our application into the Guice binding or have the Guice object accessible from our application. Is this possible?

Sorry I cannot help with Spring issues.

-- 
Andrew Gaul
http://gaul.org/