You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@jclouds.apache.org by Roman Coedo <no...@github.com> on 2015/08/18 09:03:10 UTC
[jclouds] JCLOUDS-826: Remove filesystem (#846)
You can view, comment on, or merge this pull request online at:
https://github.com/jclouds/jclouds/pull/846
-- Commit Summary --
* JCLOUDS-826: Remove filesystem
-- File Changes --
M allblobstore/pom.xml (15)
D apis/filesystem/README.txt (6)
D apis/filesystem/pom.xml (120)
D apis/filesystem/src/main/java/org/jclouds/filesystem/FilesystemApiMetadata.java (84)
D apis/filesystem/src/main/java/org/jclouds/filesystem/config/FilesystemBlobStoreContextModule.java (50)
D apis/filesystem/src/main/java/org/jclouds/filesystem/predicates/validators/FilesystemBlobKeyValidator.java (28)
D apis/filesystem/src/main/java/org/jclouds/filesystem/predicates/validators/FilesystemContainerNameValidator.java (28)
D apis/filesystem/src/main/java/org/jclouds/filesystem/predicates/validators/internal/FilesystemBlobKeyValidatorImpl.java (46)
D apis/filesystem/src/main/java/org/jclouds/filesystem/predicates/validators/internal/FilesystemContainerNameValidatorImpl.java (46)
D apis/filesystem/src/main/java/org/jclouds/filesystem/reference/FilesystemConstants.java (33)
D apis/filesystem/src/main/java/org/jclouds/filesystem/strategy/internal/FilesystemStorageStrategyImpl.java (865)
D apis/filesystem/src/main/java/org/jclouds/filesystem/util/Utils.java (114)
D apis/filesystem/src/main/java/org/jclouds/filesystem/util/internal/FileSystemBlobUtilsImpl.java (76)
D apis/filesystem/src/test/java/org/jclouds/filesystem/FilesystemApiMetadataTest.java (28)
D apis/filesystem/src/test/java/org/jclouds/filesystem/FilesystemBlobStoreTest.java (912)
D apis/filesystem/src/test/java/org/jclouds/filesystem/integration/FilesystemBlobIntegrationTest.java (88)
D apis/filesystem/src/test/java/org/jclouds/filesystem/integration/FilesystemContainerIntegrationTest.java (175)
D apis/filesystem/src/test/java/org/jclouds/filesystem/integration/FilesystemServiceIntegrationTestDisabled.java (40)
D apis/filesystem/src/test/java/org/jclouds/filesystem/predicates/validators/internal/FilesystemBlobKeyValidatorTest.java (59)
D apis/filesystem/src/test/java/org/jclouds/filesystem/predicates/validators/internal/FilesystemContainerNameValidatorTest.java (68)
D apis/filesystem/src/test/java/org/jclouds/filesystem/strategy/internal/FilesystemStorageStrategyImplTest.java (669)
D apis/filesystem/src/test/java/org/jclouds/filesystem/utils/TestUtils.java (227)
D apis/filesystem/src/test/resources/logging.properties (27)
M apis/pom.xml (1)
M blobstore/src/main/clojure/org/jclouds/blobstore2.clj (2)
-- Patch Links --
https://github.com/jclouds/jclouds/pull/846.patch
https://github.com/jclouds/jclouds/pull/846.diff
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Zack Shoylev <no...@github.com>.
Closed #846.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#event-388780327
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Andrew Phillips <no...@github.com>.
For what it's worth, I would much rather have a discussion at this point about **adding** an additional blobstore implementation (and then testing it out to see how useful it actually is) than about **removing** the filesystem blobstore.
The latter certainly has its disadvantages, and I think we need to communicate clearly what they are. But as far as I can see that doesn't mean that it's "useless", "fundamentally broken" or "needs to be removed."
A database-based blobstore is an interesting alternative that we should certainly experiment with (I wonder whether we'll see a variant based on Docker data containers - @andreaturli ping? ;-)), but requiring a database **and** an object mapper as we currently do significantly raises the convenience bar for usage.
I would certainly not say that the current JPA blobstore, as far as I understand it, is a "drop-in replacement" for the filesystem blobstore at present.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#issuecomment-133193405
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Roman Coedo <no...@github.com>.
This PR was opened to discuss the removal of filesystem with the community, which was one of the points on my GSoC proposal.
When I started the JDBC Blobstore one of the requisites was portability between different database technologies. Also, we decided to store the blobs in the database. Unfortunately, the blob storage apis offered by databases are not standard, so it's really hard to offer a generic jdbc blobstore without an object mapper. We ended up with something closer to a JPA Blobstore instead of a JDBC blobstore.
IMHO jdbc blobstore and filesystem blobstore are different things and both should stay. Maybe in the future we can come up with an hybrid database which stores the blobs in the filesystem and the metadata in an embedded database. Storing only metadata in the database would make way easier to implement the jdbc calls in a generic way. It would be a better replacement.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#issuecomment-132174234
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Zack Shoylev <no...@github.com>.
Thank you all for the feedback! With the (big) exception of managing existing files, I think a drop-in replacement should be more-or-less possible functionally (I am not sure what other limitations there are).
The current filesystem metadata implementation, while limited, works. JDBC is a nice alternative to have. I am not so sure about implementing a hybrid - why not just use the JDBC one if you need more metadata?
I will be closing this PR. It seems we should focus on promoting JDBC to the main repo for now. I still think long-term the JDBC provider should prove to be more resilient - better metadata storage and no recurring cross-platform problems, but perhaps a full replacement is not reasonable at this point.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#issuecomment-133422467
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Ignasi Barrera <no...@github.com>.
The issue basically tries to address the limitation that some filesystems are unable to store the metadata. Personally I don't see the jdbc as a replacement for the filesystem blobstore, but I also understand it can be useful in some test scenarios. I'd like to get jdbc promoted to the main repo but I think the filesystem provider comes in handy in many situations and we should try to keep it.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#issuecomment-133188665
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Andrew Gaul <no...@github.com>.
Closed #846.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#event-385175794
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Ka-Hing Cheung <no...@github.com>.
The filesystem blobstore has the advantage of working with existing files. There maybe some advantages of a jdbc only blobstore, but I think people will vastly prefer one that is closer to the existing filesystem blobstore. The optimization that @rcoedo suggested (storing metadata in a database) is a great idea for the filesystem blobstore.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#issuecomment-133113001
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Zack Shoylev <no...@github.com>.
Reopened #846.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#event-385817737
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Zack Shoylev <no...@github.com>.
@nacx ping as well.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#issuecomment-132348106
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Zack Shoylev <no...@github.com>.
Here is a tentative plan:
1. Promote jdbc to jclouds.
2. Keep both jdbc and filesystem for a while, transition to the jdbc API in projects.
3. Eventually figure out if we want to keep filesystem (maybe consider this for 2.0 or after).
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#issuecomment-132349081
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Andrew Gaul <no...@github.com>.
:-1: The filesystem blobstore is useful to present existing data via a blobstore interface as well as for larger tests. Further I have existing users of it via S3Proxy.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#issuecomment-132148614
Re: [jclouds] JCLOUDS-826: Remove filesystem (#846)
Posted by Zack Shoylev <no...@github.com>.
@andrewgaul Let's discuss this a bit more. The jdbc is supposed to be a simpler os-neutral implementation and replacement of filesystem. It should have the same capabilities. Is there a problem with the underlying idea?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/846#issuecomment-132348058