You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Carsten Ziegeler <cz...@apache.org> on 2018/11/11 09:55:42 UTC
[RT] Simple File System Resource Provider
I've recently tried to run a minimal Sling without JCR. Obviously you
need at least one resource provider to have some content, so I picked
the easiest choice, the file system resource provider.
However, it turned out that this is not the easiest choice for this
scenario as it has a lot of features, especially support for handling
content XML files and vault files, which again brings in the whole
dependency list to jcr related bundles. In my case I just want to stream
binaries and json files, so none of the above is needed. But still I
need to deploy all the bundles. In addition there are other things like
the json parsing library is embedded in the bundle etc.
Now, I think we should really have a simple file system resource
provider which only does the basics and has not an endless list of
bundles. I see two ways to get there: make the current provider
extensible and provide all this extra cruft as extensions or write a new
simple provider.
Thoughts?
Regards
Carsten
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: [RT] Simple File System Resource Provider
Posted by Roy Teeuwen <ro...@teeuwen.be>.
Seeing as the current filesystem resource provider does need an update to java 8 api, it would be nice to refactor / start over with that one and make it more generic / extensible? It has some caveats (like the start up scanning etc.) that would be nice to have fixed in a more elegant way.
Roy
> On 11 Nov 2018, at 10:55, Carsten Ziegeler <cz...@apache.org> wrote:
>
> I've recently tried to run a minimal Sling without JCR. Obviously you need at least one resource provider to have some content, so I picked the easiest choice, the file system resource provider.
>
> However, it turned out that this is not the easiest choice for this scenario as it has a lot of features, especially support for handling content XML files and vault files, which again brings in the whole dependency list to jcr related bundles. In my case I just want to stream binaries and json files, so none of the above is needed. But still I need to deploy all the bundles. In addition there are other things like the json parsing library is embedded in the bundle etc.
>
> Now, I think we should really have a simple file system resource provider which only does the basics and has not an endless list of bundles. I see two ways to get there: make the current provider extensible and provide all this extra cruft as extensions or write a new simple provider.
>
> Thoughts?
>
> Regards
> Carsten
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziegeler@apache.org
Re: [RT] Simple File System Resource Provider
Posted by Julian Sedding <js...@gmail.com>.
+1 for creating a "simple" FS resource provider with _pluggable_
formats. Maybe it doesn't even need to be FS specific? Just "simple"
:)
Things like a mock JCR implementation are very useful for some
use-cases, but IMHO they could also be achieved with a custom adapter
factory and thus be highly decoupled.
Resource event support is probably harder to decouple, but could be
configurable (on/off).
When I last used the FS RP, I found that it was lacking stable child
iteration order - I think thjis may be desirable for a new "simple" RP
as well. For the FS RP I made this alphabetical for file-based
resources. Resources defined withion an XML/JSON file retain the order
in which they are read.
Regards
Julian
On Mon, Nov 12, 2018 at 3:13 PM Jason E Bailey <je...@apache.org> wrote:
>
> +1 on a simple resource provider, I'm curious about the focus on a filesystem provider. I've started playing around with creating one and I see a clear logical association between what a resource provider needs and a key-value store. Good key-value stores already have versioning, and non blocking read and writes. Things that are architecturally nice to have removed from our hands.
>
> Something like mvstore - http://www.h2database.com/html/mvstore.html
> or lmdb - https://symas.com/lmdb/
>
> would be really good to build on.
>
> - Jason
>
> On Sun, Nov 11, 2018, at 4:55 AM, Carsten Ziegeler wrote:
> > I've recently tried to run a minimal Sling without JCR. Obviously you
> > need at least one resource provider to have some content, so I picked
> > the easiest choice, the file system resource provider.
> >
> > However, it turned out that this is not the easiest choice for this
> > scenario as it has a lot of features, especially support for handling
> > content XML files and vault files, which again brings in the whole
> > dependency list to jcr related bundles. In my case I just want to stream
> > binaries and json files, so none of the above is needed. But still I
> > need to deploy all the bundles. In addition there are other things like
> > the json parsing library is embedded in the bundle etc.
> >
> > Now, I think we should really have a simple file system resource
> > provider which only does the basics and has not an endless list of
> > bundles. I see two ways to get there: make the current provider
> > extensible and provide all this extra cruft as extensions or write a new
> > simple provider.
> >
> > Thoughts?
> >
> > Regards
> > Carsten
> > --
> > Carsten Ziegeler
> > Adobe Research Switzerland
> > cziegeler@apache.org
Re: [RT] Simple File System Resource Provider
Posted by Jason E Bailey <je...@apache.org>.
+1 on a simple resource provider, I'm curious about the focus on a filesystem provider. I've started playing around with creating one and I see a clear logical association between what a resource provider needs and a key-value store. Good key-value stores already have versioning, and non blocking read and writes. Things that are architecturally nice to have removed from our hands.
Something like mvstore - http://www.h2database.com/html/mvstore.html
or lmdb - https://symas.com/lmdb/
would be really good to build on.
- Jason
On Sun, Nov 11, 2018, at 4:55 AM, Carsten Ziegeler wrote:
> I've recently tried to run a minimal Sling without JCR. Obviously you
> need at least one resource provider to have some content, so I picked
> the easiest choice, the file system resource provider.
>
> However, it turned out that this is not the easiest choice for this
> scenario as it has a lot of features, especially support for handling
> content XML files and vault files, which again brings in the whole
> dependency list to jcr related bundles. In my case I just want to stream
> binaries and json files, so none of the above is needed. But still I
> need to deploy all the bundles. In addition there are other things like
> the json parsing library is embedded in the bundle etc.
>
> Now, I think we should really have a simple file system resource
> provider which only does the basics and has not an endless list of
> bundles. I see two ways to get there: make the current provider
> extensible and provide all this extra cruft as extensions or write a new
> simple provider.
>
> Thoughts?
>
> Regards
> Carsten
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziegeler@apache.org
Re: [RT] Simple File System Resource Provider
Posted by Carsten Ziegeler <cz...@apache.org>.
Am 12.11.2018 um 14:31 schrieb Bertrand Delacretaz:
> Hi,
>
> On Mon, Nov 12, 2018 at 12:20 PM Carsten Ziegeler <cz...@apache.org> wrote:
>> ...As a side note, the markdown resource provider in the whiteboard is
>> another file system based rp which is closer to my use case, except that
>> it supports markdown files instead of json...
>
> Having a GitHub-friendly resource provider would be cool IMO, and that
> one looks like a good option for that.
>
Absolutely, but it's mixing concerns, it mixes where to fetch resources
from (file system) with the type of resources (markdown files). If my md
files are somewhere else I can't use that RP. And if I have mixed files
in my file system it might get challenging as well.
Ultimately, the current fs provider has the same problem.
Carsten
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: [RT] Simple File System Resource Provider
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi,
On Mon, Nov 12, 2018 at 12:20 PM Carsten Ziegeler <cz...@apache.org> wrote:
> ...As a side note, the markdown resource provider in the whiteboard is
> another file system based rp which is closer to my use case, except that
> it supports markdown files instead of json...
Having a GitHub-friendly resource provider would be cool IMO, and that
one looks like a good option for that.
-Bertrand
Re: [RT] Simple File System Resource Provider
Posted by Carsten Ziegeler <cz...@apache.org>.
Yes, I was thinking of using the 1.x branch, however that branch is at
version 1.3.x which contains a lot of the unneeded (from my pov) as well.
Maybe I just start a prototype in the whiteboard and then we can figure
out what to do about it :)
As a side note, the markdown resource provider in the whiteboard is
another file system based rp which is closer to my use case, except that
it supports markdown files instead of json.
Regards
Carsten
Am 12.11.2018 um 12:03 schrieb Stefan Seifert:
> ah, ok - this sounds like fsresource 1.2.2 (all the new stuff was added in 2.x).
> in this case it might make sense to create a new bundle without the existing one, or maintain the 1.x branch in parallel to 2.x or fork it to a new bundle.
>
> stefan
>
>> -----Original Message-----
>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>> Sent: Monday, November 12, 2018 11:51 AM
>> To: dev@sling.apache.org; Stefan Seifert
>> Subject: Re: [RT] Simple File System Resource Provider
>>
>> It's only 1 - 2 from your list is taking a structured json and creates
>> resources out of the structure (if I'm not mistaken) which is something
>> I don't need.
>>
>> With the approach I hope there is no need for b) (caching) as the
>> mapping is 1:1 (more or less). As there is no need for cachign I don't
>> think a) is needed either.
>>
>> Regards
>> Carsten
>>
>> Am 12.11.2018 um 11:23 schrieb Stefan Seifert:
>>> so from my list below your use case is 1. + 2. + a) + probably b) and
>> leaving out the other parts?
>>> or something different from what is implemented currently?
>>>
>>> stefan
>>>
>>>> -----Original Message-----
>>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>>>> Sent: Monday, November 12, 2018 11:20 AM
>>>> To: dev@sling.apache.org; Stefan Seifert
>>>> Subject: Re: [RT] Simple File System Resource Provider
>>>>
>>>> Hi,
>>>>
>>>> I would be totally happy if we can factor out the extensions, I'm
>>>> wondering however if this is worth the effort.
>>>>
>>>> In my use case, I would like to have a simple mapping to directories and
>>>> files, supporting json and binary files. So a resource maps to a json
>>>> file 1:1 regardless of the structure of the json file and a such a
>>>> resource can have an additional binary.
>>>>
>>>> I understand the need for the support of all the other features we have
>>>> today, but they are not needed for other use cases.
>>>>
>>>> Regards
>>>> Carsten
>>>>
>>>> Am 12.11.2018 um 11:06 schrieb Stefan Seifert:
>>>>> yes, the current implementation of the fsresource provider is no longer
>>>> any simple.
>>>>>
>>>>> it currently supports three (configurable) modes:
>>>>> 1. simple mapping of folders and binary files from filesystem (this was
>>>> the starting point of fsresource)
>>>>> 2. reading structured resource data from JSON files and folders in the
>>>> same way it is done by the content loader
>>>>> 3. reading structured resource data from FileVault XML files as it's
>>>> stored in content packages
>>>>>
>>>>> and features:
>>>>> a) sending resource events if any of these files are changed in runtime
>>>>> b) implement some caching to speed things up
>>>>> c) support not only the Sling Resource API, but also simulate an
>>>> underlying JCR API for code that runs on top which is still using the
>> JCR
>>>> API for cases where also the Sling Resource API would suffice but cannot
>> be
>>>> changed because it's part of a product...
>>>>>
>>>>> so the use case ranges from simple mapping of a bunch of static files
>> to
>>>> full-blown emulation of a JCR repository out of a complex project
>> structure
>>>> in the filesystem e.g. for usage in a development environemnt (see [0]).
>>>>>
>>>>> ---
>>>>>
>>>>> removing the embedded json libraries should be simple, it was only for
>>>> convenience when the fsresource bundle is to deployed afterwards to an
>>>> existing instance.
>>>>> but the dependencies to all those JCR-related bundles remains as long
>> as
>>>> all three modes and features need to be supported.
>>>>>
>>>>> i'm not sure if implementing a new fsresource provider e.g. only for
>>>> 1.+2. from scratch would be the best way. there is a lot of special
>> logic
>>>> for edge cases esp. in 2. to make sure it behaves the same as the
>> content
>>>> loader which we have then to duplicate another time. it should be
>> possible
>>>> to split the existing fsresource into a core and extension bundle as
>> it's
>>>> somewhat separated already due to the different supported modes
>> 1./2./3.,
>>>> and the virtual JCR API support could be made configurable as well.
>>>>>
>>>>> supporting Java 8 features for the filesystem changes detection would
>> be
>>>> a good thing; last time i was looking into it i failed to make good use
>> of
>>>> it due to strange implementation differences on windows and unix file
>>>> systems (and those differences where covered by the JavaDocs). but maybe
>>>> there is a way to do it right.
>>>>>
>>>>> stefan
>>>>>
>>>>> [0] https://adapt.to/2017/en/schedule/ease-development-with-apache-
>> sling-
>>>> file-system-resource-provider.html
>>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>>>>>> Sent: Sunday, November 11, 2018 10:56 AM
>>>>>> To: Sling Developers
>>>>>> Subject: [RT] Simple File System Resource Provider
>>>>>>
>>>>>> I've recently tried to run a minimal Sling without JCR. Obviously you
>>>>>> need at least one resource provider to have some content, so I picked
>>>>>> the easiest choice, the file system resource provider.
>>>>>>
>>>>>> However, it turned out that this is not the easiest choice for this
>>>>>> scenario as it has a lot of features, especially support for handling
>>>>>> content XML files and vault files, which again brings in the whole
>>>>>> dependency list to jcr related bundles. In my case I just want to
>> stream
>>>>>> binaries and json files, so none of the above is needed. But still I
>>>>>> need to deploy all the bundles. In addition there are other things
>> like
>>>>>> the json parsing library is embedded in the bundle etc.
>>>>>>
>>>>>> Now, I think we should really have a simple file system resource
>>>>>> provider which only does the basics and has not an endless list of
>>>>>> bundles. I see two ways to get there: make the current provider
>>>>>> extensible and provide all this extra cruft as extensions or write a
>> new
>>>>>> simple provider.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> Regards
>>>>>> Carsten
>>>>>> --
>>>>>> Carsten Ziegeler
>>>>>> Adobe Research Switzerland
>>>>>> cziegeler@apache.org
>>>>>
>>>>
>>>> --
>>>> Carsten Ziegeler
>>>> Adobe Research Switzerland
>>>> cziegeler@apache.org
>>>
>>
>> --
>> Carsten Ziegeler
>> Adobe Research Switzerland
>> cziegeler@apache.org
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
RE: [RT] Simple File System Resource Provider
Posted by Stefan Seifert <ss...@pro-vision.de>.
ah, ok - this sounds like fsresource 1.2.2 (all the new stuff was added in 2.x).
in this case it might make sense to create a new bundle without the existing one, or maintain the 1.x branch in parallel to 2.x or fork it to a new bundle.
stefan
>-----Original Message-----
>From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>Sent: Monday, November 12, 2018 11:51 AM
>To: dev@sling.apache.org; Stefan Seifert
>Subject: Re: [RT] Simple File System Resource Provider
>
>It's only 1 - 2 from your list is taking a structured json and creates
>resources out of the structure (if I'm not mistaken) which is something
>I don't need.
>
>With the approach I hope there is no need for b) (caching) as the
>mapping is 1:1 (more or less). As there is no need for cachign I don't
>think a) is needed either.
>
>Regards
>Carsten
>
>Am 12.11.2018 um 11:23 schrieb Stefan Seifert:
>> so from my list below your use case is 1. + 2. + a) + probably b) and
>leaving out the other parts?
>> or something different from what is implemented currently?
>>
>> stefan
>>
>>> -----Original Message-----
>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>>> Sent: Monday, November 12, 2018 11:20 AM
>>> To: dev@sling.apache.org; Stefan Seifert
>>> Subject: Re: [RT] Simple File System Resource Provider
>>>
>>> Hi,
>>>
>>> I would be totally happy if we can factor out the extensions, I'm
>>> wondering however if this is worth the effort.
>>>
>>> In my use case, I would like to have a simple mapping to directories and
>>> files, supporting json and binary files. So a resource maps to a json
>>> file 1:1 regardless of the structure of the json file and a such a
>>> resource can have an additional binary.
>>>
>>> I understand the need for the support of all the other features we have
>>> today, but they are not needed for other use cases.
>>>
>>> Regards
>>> Carsten
>>>
>>> Am 12.11.2018 um 11:06 schrieb Stefan Seifert:
>>>> yes, the current implementation of the fsresource provider is no longer
>>> any simple.
>>>>
>>>> it currently supports three (configurable) modes:
>>>> 1. simple mapping of folders and binary files from filesystem (this was
>>> the starting point of fsresource)
>>>> 2. reading structured resource data from JSON files and folders in the
>>> same way it is done by the content loader
>>>> 3. reading structured resource data from FileVault XML files as it's
>>> stored in content packages
>>>>
>>>> and features:
>>>> a) sending resource events if any of these files are changed in runtime
>>>> b) implement some caching to speed things up
>>>> c) support not only the Sling Resource API, but also simulate an
>>> underlying JCR API for code that runs on top which is still using the
>JCR
>>> API for cases where also the Sling Resource API would suffice but cannot
>be
>>> changed because it's part of a product...
>>>>
>>>> so the use case ranges from simple mapping of a bunch of static files
>to
>>> full-blown emulation of a JCR repository out of a complex project
>structure
>>> in the filesystem e.g. for usage in a development environemnt (see [0]).
>>>>
>>>> ---
>>>>
>>>> removing the embedded json libraries should be simple, it was only for
>>> convenience when the fsresource bundle is to deployed afterwards to an
>>> existing instance.
>>>> but the dependencies to all those JCR-related bundles remains as long
>as
>>> all three modes and features need to be supported.
>>>>
>>>> i'm not sure if implementing a new fsresource provider e.g. only for
>>> 1.+2. from scratch would be the best way. there is a lot of special
>logic
>>> for edge cases esp. in 2. to make sure it behaves the same as the
>content
>>> loader which we have then to duplicate another time. it should be
>possible
>>> to split the existing fsresource into a core and extension bundle as
>it's
>>> somewhat separated already due to the different supported modes
>1./2./3.,
>>> and the virtual JCR API support could be made configurable as well.
>>>>
>>>> supporting Java 8 features for the filesystem changes detection would
>be
>>> a good thing; last time i was looking into it i failed to make good use
>of
>>> it due to strange implementation differences on windows and unix file
>>> systems (and those differences where covered by the JavaDocs). but maybe
>>> there is a way to do it right.
>>>>
>>>> stefan
>>>>
>>>> [0] https://adapt.to/2017/en/schedule/ease-development-with-apache-
>sling-
>>> file-system-resource-provider.html
>>>>
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>>>>> Sent: Sunday, November 11, 2018 10:56 AM
>>>>> To: Sling Developers
>>>>> Subject: [RT] Simple File System Resource Provider
>>>>>
>>>>> I've recently tried to run a minimal Sling without JCR. Obviously you
>>>>> need at least one resource provider to have some content, so I picked
>>>>> the easiest choice, the file system resource provider.
>>>>>
>>>>> However, it turned out that this is not the easiest choice for this
>>>>> scenario as it has a lot of features, especially support for handling
>>>>> content XML files and vault files, which again brings in the whole
>>>>> dependency list to jcr related bundles. In my case I just want to
>stream
>>>>> binaries and json files, so none of the above is needed. But still I
>>>>> need to deploy all the bundles. In addition there are other things
>like
>>>>> the json parsing library is embedded in the bundle etc.
>>>>>
>>>>> Now, I think we should really have a simple file system resource
>>>>> provider which only does the basics and has not an endless list of
>>>>> bundles. I see two ways to get there: make the current provider
>>>>> extensible and provide all this extra cruft as extensions or write a
>new
>>>>> simple provider.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Regards
>>>>> Carsten
>>>>> --
>>>>> Carsten Ziegeler
>>>>> Adobe Research Switzerland
>>>>> cziegeler@apache.org
>>>>
>>>
>>> --
>>> Carsten Ziegeler
>>> Adobe Research Switzerland
>>> cziegeler@apache.org
>>
>
>--
>Carsten Ziegeler
>Adobe Research Switzerland
>cziegeler@apache.org
Re: [RT] Simple File System Resource Provider
Posted by Carsten Ziegeler <cz...@apache.org>.
It's only 1 - 2 from your list is taking a structured json and creates
resources out of the structure (if I'm not mistaken) which is something
I don't need.
With the approach I hope there is no need for b) (caching) as the
mapping is 1:1 (more or less). As there is no need for cachign I don't
think a) is needed either.
Regards
Carsten
Am 12.11.2018 um 11:23 schrieb Stefan Seifert:
> so from my list below your use case is 1. + 2. + a) + probably b) and leaving out the other parts?
> or something different from what is implemented currently?
>
> stefan
>
>> -----Original Message-----
>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>> Sent: Monday, November 12, 2018 11:20 AM
>> To: dev@sling.apache.org; Stefan Seifert
>> Subject: Re: [RT] Simple File System Resource Provider
>>
>> Hi,
>>
>> I would be totally happy if we can factor out the extensions, I'm
>> wondering however if this is worth the effort.
>>
>> In my use case, I would like to have a simple mapping to directories and
>> files, supporting json and binary files. So a resource maps to a json
>> file 1:1 regardless of the structure of the json file and a such a
>> resource can have an additional binary.
>>
>> I understand the need for the support of all the other features we have
>> today, but they are not needed for other use cases.
>>
>> Regards
>> Carsten
>>
>> Am 12.11.2018 um 11:06 schrieb Stefan Seifert:
>>> yes, the current implementation of the fsresource provider is no longer
>> any simple.
>>>
>>> it currently supports three (configurable) modes:
>>> 1. simple mapping of folders and binary files from filesystem (this was
>> the starting point of fsresource)
>>> 2. reading structured resource data from JSON files and folders in the
>> same way it is done by the content loader
>>> 3. reading structured resource data from FileVault XML files as it's
>> stored in content packages
>>>
>>> and features:
>>> a) sending resource events if any of these files are changed in runtime
>>> b) implement some caching to speed things up
>>> c) support not only the Sling Resource API, but also simulate an
>> underlying JCR API for code that runs on top which is still using the JCR
>> API for cases where also the Sling Resource API would suffice but cannot be
>> changed because it's part of a product...
>>>
>>> so the use case ranges from simple mapping of a bunch of static files to
>> full-blown emulation of a JCR repository out of a complex project structure
>> in the filesystem e.g. for usage in a development environemnt (see [0]).
>>>
>>> ---
>>>
>>> removing the embedded json libraries should be simple, it was only for
>> convenience when the fsresource bundle is to deployed afterwards to an
>> existing instance.
>>> but the dependencies to all those JCR-related bundles remains as long as
>> all three modes and features need to be supported.
>>>
>>> i'm not sure if implementing a new fsresource provider e.g. only for
>> 1.+2. from scratch would be the best way. there is a lot of special logic
>> for edge cases esp. in 2. to make sure it behaves the same as the content
>> loader which we have then to duplicate another time. it should be possible
>> to split the existing fsresource into a core and extension bundle as it's
>> somewhat separated already due to the different supported modes 1./2./3.,
>> and the virtual JCR API support could be made configurable as well.
>>>
>>> supporting Java 8 features for the filesystem changes detection would be
>> a good thing; last time i was looking into it i failed to make good use of
>> it due to strange implementation differences on windows and unix file
>> systems (and those differences where covered by the JavaDocs). but maybe
>> there is a way to do it right.
>>>
>>> stefan
>>>
>>> [0] https://adapt.to/2017/en/schedule/ease-development-with-apache-sling-
>> file-system-resource-provider.html
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>>>> Sent: Sunday, November 11, 2018 10:56 AM
>>>> To: Sling Developers
>>>> Subject: [RT] Simple File System Resource Provider
>>>>
>>>> I've recently tried to run a minimal Sling without JCR. Obviously you
>>>> need at least one resource provider to have some content, so I picked
>>>> the easiest choice, the file system resource provider.
>>>>
>>>> However, it turned out that this is not the easiest choice for this
>>>> scenario as it has a lot of features, especially support for handling
>>>> content XML files and vault files, which again brings in the whole
>>>> dependency list to jcr related bundles. In my case I just want to stream
>>>> binaries and json files, so none of the above is needed. But still I
>>>> need to deploy all the bundles. In addition there are other things like
>>>> the json parsing library is embedded in the bundle etc.
>>>>
>>>> Now, I think we should really have a simple file system resource
>>>> provider which only does the basics and has not an endless list of
>>>> bundles. I see two ways to get there: make the current provider
>>>> extensible and provide all this extra cruft as extensions or write a new
>>>> simple provider.
>>>>
>>>> Thoughts?
>>>>
>>>> Regards
>>>> Carsten
>>>> --
>>>> Carsten Ziegeler
>>>> Adobe Research Switzerland
>>>> cziegeler@apache.org
>>>
>>
>> --
>> Carsten Ziegeler
>> Adobe Research Switzerland
>> cziegeler@apache.org
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
RE: [RT] Simple File System Resource Provider
Posted by Stefan Seifert <ss...@pro-vision.de>.
so from my list below your use case is 1. + 2. + a) + probably b) and leaving out the other parts?
or something different from what is implemented currently?
stefan
>-----Original Message-----
>From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>Sent: Monday, November 12, 2018 11:20 AM
>To: dev@sling.apache.org; Stefan Seifert
>Subject: Re: [RT] Simple File System Resource Provider
>
>Hi,
>
>I would be totally happy if we can factor out the extensions, I'm
>wondering however if this is worth the effort.
>
>In my use case, I would like to have a simple mapping to directories and
>files, supporting json and binary files. So a resource maps to a json
>file 1:1 regardless of the structure of the json file and a such a
>resource can have an additional binary.
>
>I understand the need for the support of all the other features we have
>today, but they are not needed for other use cases.
>
>Regards
>Carsten
>
>Am 12.11.2018 um 11:06 schrieb Stefan Seifert:
>> yes, the current implementation of the fsresource provider is no longer
>any simple.
>>
>> it currently supports three (configurable) modes:
>> 1. simple mapping of folders and binary files from filesystem (this was
>the starting point of fsresource)
>> 2. reading structured resource data from JSON files and folders in the
>same way it is done by the content loader
>> 3. reading structured resource data from FileVault XML files as it's
>stored in content packages
>>
>> and features:
>> a) sending resource events if any of these files are changed in runtime
>> b) implement some caching to speed things up
>> c) support not only the Sling Resource API, but also simulate an
>underlying JCR API for code that runs on top which is still using the JCR
>API for cases where also the Sling Resource API would suffice but cannot be
>changed because it's part of a product...
>>
>> so the use case ranges from simple mapping of a bunch of static files to
>full-blown emulation of a JCR repository out of a complex project structure
>in the filesystem e.g. for usage in a development environemnt (see [0]).
>>
>> ---
>>
>> removing the embedded json libraries should be simple, it was only for
>convenience when the fsresource bundle is to deployed afterwards to an
>existing instance.
>> but the dependencies to all those JCR-related bundles remains as long as
>all three modes and features need to be supported.
>>
>> i'm not sure if implementing a new fsresource provider e.g. only for
>1.+2. from scratch would be the best way. there is a lot of special logic
>for edge cases esp. in 2. to make sure it behaves the same as the content
>loader which we have then to duplicate another time. it should be possible
>to split the existing fsresource into a core and extension bundle as it's
>somewhat separated already due to the different supported modes 1./2./3.,
>and the virtual JCR API support could be made configurable as well.
>>
>> supporting Java 8 features for the filesystem changes detection would be
>a good thing; last time i was looking into it i failed to make good use of
>it due to strange implementation differences on windows and unix file
>systems (and those differences where covered by the JavaDocs). but maybe
>there is a way to do it right.
>>
>> stefan
>>
>> [0] https://adapt.to/2017/en/schedule/ease-development-with-apache-sling-
>file-system-resource-provider.html
>>
>>
>>
>>> -----Original Message-----
>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>>> Sent: Sunday, November 11, 2018 10:56 AM
>>> To: Sling Developers
>>> Subject: [RT] Simple File System Resource Provider
>>>
>>> I've recently tried to run a minimal Sling without JCR. Obviously you
>>> need at least one resource provider to have some content, so I picked
>>> the easiest choice, the file system resource provider.
>>>
>>> However, it turned out that this is not the easiest choice for this
>>> scenario as it has a lot of features, especially support for handling
>>> content XML files and vault files, which again brings in the whole
>>> dependency list to jcr related bundles. In my case I just want to stream
>>> binaries and json files, so none of the above is needed. But still I
>>> need to deploy all the bundles. In addition there are other things like
>>> the json parsing library is embedded in the bundle etc.
>>>
>>> Now, I think we should really have a simple file system resource
>>> provider which only does the basics and has not an endless list of
>>> bundles. I see two ways to get there: make the current provider
>>> extensible and provide all this extra cruft as extensions or write a new
>>> simple provider.
>>>
>>> Thoughts?
>>>
>>> Regards
>>> Carsten
>>> --
>>> Carsten Ziegeler
>>> Adobe Research Switzerland
>>> cziegeler@apache.org
>>
>
>--
>Carsten Ziegeler
>Adobe Research Switzerland
>cziegeler@apache.org
Re: [RT] Simple File System Resource Provider
Posted by Carsten Ziegeler <cz...@apache.org>.
Hi,
I would be totally happy if we can factor out the extensions, I'm
wondering however if this is worth the effort.
In my use case, I would like to have a simple mapping to directories and
files, supporting json and binary files. So a resource maps to a json
file 1:1 regardless of the structure of the json file and a such a
resource can have an additional binary.
I understand the need for the support of all the other features we have
today, but they are not needed for other use cases.
Regards
Carsten
Am 12.11.2018 um 11:06 schrieb Stefan Seifert:
> yes, the current implementation of the fsresource provider is no longer any simple.
>
> it currently supports three (configurable) modes:
> 1. simple mapping of folders and binary files from filesystem (this was the starting point of fsresource)
> 2. reading structured resource data from JSON files and folders in the same way it is done by the content loader
> 3. reading structured resource data from FileVault XML files as it's stored in content packages
>
> and features:
> a) sending resource events if any of these files are changed in runtime
> b) implement some caching to speed things up
> c) support not only the Sling Resource API, but also simulate an underlying JCR API for code that runs on top which is still using the JCR API for cases where also the Sling Resource API would suffice but cannot be changed because it's part of a product...
>
> so the use case ranges from simple mapping of a bunch of static files to full-blown emulation of a JCR repository out of a complex project structure in the filesystem e.g. for usage in a development environemnt (see [0]).
>
> ---
>
> removing the embedded json libraries should be simple, it was only for convenience when the fsresource bundle is to deployed afterwards to an existing instance.
> but the dependencies to all those JCR-related bundles remains as long as all three modes and features need to be supported.
>
> i'm not sure if implementing a new fsresource provider e.g. only for 1.+2. from scratch would be the best way. there is a lot of special logic for edge cases esp. in 2. to make sure it behaves the same as the content loader which we have then to duplicate another time. it should be possible to split the existing fsresource into a core and extension bundle as it's somewhat separated already due to the different supported modes 1./2./3., and the virtual JCR API support could be made configurable as well.
>
> supporting Java 8 features for the filesystem changes detection would be a good thing; last time i was looking into it i failed to make good use of it due to strange implementation differences on windows and unix file systems (and those differences where covered by the JavaDocs). but maybe there is a way to do it right.
>
> stefan
>
> [0] https://adapt.to/2017/en/schedule/ease-development-with-apache-sling-file-system-resource-provider.html
>
>
>
>> -----Original Message-----
>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>> Sent: Sunday, November 11, 2018 10:56 AM
>> To: Sling Developers
>> Subject: [RT] Simple File System Resource Provider
>>
>> I've recently tried to run a minimal Sling without JCR. Obviously you
>> need at least one resource provider to have some content, so I picked
>> the easiest choice, the file system resource provider.
>>
>> However, it turned out that this is not the easiest choice for this
>> scenario as it has a lot of features, especially support for handling
>> content XML files and vault files, which again brings in the whole
>> dependency list to jcr related bundles. In my case I just want to stream
>> binaries and json files, so none of the above is needed. But still I
>> need to deploy all the bundles. In addition there are other things like
>> the json parsing library is embedded in the bundle etc.
>>
>> Now, I think we should really have a simple file system resource
>> provider which only does the basics and has not an endless list of
>> bundles. I see two ways to get there: make the current provider
>> extensible and provide all this extra cruft as extensions or write a new
>> simple provider.
>>
>> Thoughts?
>>
>> Regards
>> Carsten
>> --
>> Carsten Ziegeler
>> Adobe Research Switzerland
>> cziegeler@apache.org
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
RE: [RT] Simple File System Resource Provider
Posted by Stefan Seifert <ss...@pro-vision.de>.
yes, the current implementation of the fsresource provider is no longer any simple.
it currently supports three (configurable) modes:
1. simple mapping of folders and binary files from filesystem (this was the starting point of fsresource)
2. reading structured resource data from JSON files and folders in the same way it is done by the content loader
3. reading structured resource data from FileVault XML files as it's stored in content packages
and features:
a) sending resource events if any of these files are changed in runtime
b) implement some caching to speed things up
c) support not only the Sling Resource API, but also simulate an underlying JCR API for code that runs on top which is still using the JCR API for cases where also the Sling Resource API would suffice but cannot be changed because it's part of a product...
so the use case ranges from simple mapping of a bunch of static files to full-blown emulation of a JCR repository out of a complex project structure in the filesystem e.g. for usage in a development environemnt (see [0]).
---
removing the embedded json libraries should be simple, it was only for convenience when the fsresource bundle is to deployed afterwards to an existing instance.
but the dependencies to all those JCR-related bundles remains as long as all three modes and features need to be supported.
i'm not sure if implementing a new fsresource provider e.g. only for 1.+2. from scratch would be the best way. there is a lot of special logic for edge cases esp. in 2. to make sure it behaves the same as the content loader which we have then to duplicate another time. it should be possible to split the existing fsresource into a core and extension bundle as it's somewhat separated already due to the different supported modes 1./2./3., and the virtual JCR API support could be made configurable as well.
supporting Java 8 features for the filesystem changes detection would be a good thing; last time i was looking into it i failed to make good use of it due to strange implementation differences on windows and unix file systems (and those differences where covered by the JavaDocs). but maybe there is a way to do it right.
stefan
[0] https://adapt.to/2017/en/schedule/ease-development-with-apache-sling-file-system-resource-provider.html
>-----Original Message-----
>From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>Sent: Sunday, November 11, 2018 10:56 AM
>To: Sling Developers
>Subject: [RT] Simple File System Resource Provider
>
>I've recently tried to run a minimal Sling without JCR. Obviously you
>need at least one resource provider to have some content, so I picked
>the easiest choice, the file system resource provider.
>
>However, it turned out that this is not the easiest choice for this
>scenario as it has a lot of features, especially support for handling
>content XML files and vault files, which again brings in the whole
>dependency list to jcr related bundles. In my case I just want to stream
>binaries and json files, so none of the above is needed. But still I
>need to deploy all the bundles. In addition there are other things like
>the json parsing library is embedded in the bundle etc.
>
>Now, I think we should really have a simple file system resource
>provider which only does the basics and has not an endless list of
>bundles. I see two ways to get there: make the current provider
>extensible and provide all this extra cruft as extensions or write a new
>simple provider.
>
>Thoughts?
>
>Regards
>Carsten
>--
>Carsten Ziegeler
>Adobe Research Switzerland
>cziegeler@apache.org