[jira] Commented: (SLING-587) JcrBundlesManager

["Bertrand Delacretaz (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SLING-587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12617855#action_12617855 ] 

Bertrand Delacretaz commented on SLING-587:
-------------------------------------------

Revision 680753 contains a first usable version, here's how to test it:

a) Installing and removing bundles
Create one or several folders named "bundles" somewhere under /libs or /apps in the repository (WebDAV is fine for that). These folders must exist before the jcrbundles service starts, as the service does not yet observe the creation of new folders.

Install the jcrbundle service (currently found under http://svn.eu.apache.org/repos/asf/incubator/sling/whiteboard/jcrbundles)

Copy an OSGi bundle jar file into a bundles folder, it should be installed after a few seconds. Try http://www.knopflerfish.org/releases/2.1.0/jars/desktop_awt/desktop_awt_all-2.0.0.jar for example, which shows a nice GUI OSGi console.

Remove the bundle jar file to uninstall the bundle.

b) Creating configurations
To test the configuration feature, create a file named org.apache.sling.commons.log.LogManager.cfg, with this content, in a bundles folder:

  org.apache.sling.commons.log.file=logs/error.log
  org.apache.sling.commons.log.file.number=5
  org.apache.sling.commons.log.file.size=10mb
  org.apache.sling.commons.log.level=info
  org.apache.sling.commons.log.pattern={0,date,dd.MM.yyyy HH:mm:ss.SSS} *{4}* [{2}] {3} {5}

And watch the log level change in logs/error.log (under the Sling home directory) when changing the level parameter between "info" and "debug".

Note that the same config can be edited from the Sling console, which will show changes made in the cfg file, but the opposite is not true: changes made in the Sling console are not visible in the cfg file, that's a problem that we'll need to solve. Not to mention potential security implications - we might need to restrict what can be done with jcrbundles, or make sure that it doesn't mess with bundles and configs installed via other means.

Removing the cfg file causes the configuration to be deleted - in this case I think the log service continues to work with default parameters.

Configurations that use OSGi factories must be named like factoryName-instanceName.cfg

c) Debugging
jcrbundles logs lots of information about what it's doing, that can help troubleshooting it if needed.

d) Resync at service startup
Note that jcrbundles will resynchronize the OSGi state with what's in the bundles folders when the service starts: bundles and configs added, removed or updated while the service is not active are taken into account at the next restart.

> JcrBundlesManager
> -----------------
>
>                 Key: SLING-587
>                 URL: https://issues.apache.org/jira/browse/SLING-587
>             Project: Sling
>          Issue Type: Improvement
>          Components: OSGi
>            Reporter: Bertrand Delacretaz
>
> See http://markmail.org/message/wey5dod7cm7oalhr for a tentative spec

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: [jira] Commented: (SLING-587) JcrBundlesManager

[Felix Meschberger <fm...@gmail.com>]

Hi,

Bertrand Delacretaz schrieb:
>> ...Having said that, the jcrbundles module is free to listen for configuration
>> events from the ConfigurationAdmin to write back any configuration files...
> 
> I thought about that, but rewriting a config file that the user just
> changed might be problematic and/or surprising, so I didn't want to
> implement that in this first prototype.

Right and given, that we might not want delete any user comments, we 
might not even be able to modify the file correctly ...


> 
>> ..With bundles, it is somewhat different. Because you cannot extract the
>> bundle file of installed bundles from the framework (this is probably also
>> not what you want)...
> 
> Would it be possible for the OSGi framework to add for example an md5
> of the bundle data as an attribute somewhere? In this way we the
> various front-ends (OSGi console, jcrbundles, fileinstall, etc.) could
> decide if an update is needed or not.

A bundle has various information tags, which may be used to help in the 
decision and not all tools use all these:

   id: This is a counter maintained by the framework used only
      inside a single framework. Each concrete value is only used once
      for the entire lifetime (including start/stop cycles) of a
      framework. That is the bundle ids are not consecutive but unique.
   symbolic name: a symbolic name (ehrm, yes ;-) identifying the bundle.
      The symbolic name is made up of a limited character set.
   version: The version number of the bundle
   location: Indicating the source of the bundle installation. This
      is just a string which should be something like an URL. It is not
      required to be a real URL, though.
   lastModification: The time the last status change happend to the
      bundle. This is start/stop, install, uninstall, update ... This
      timestampe is managed by the framework.


There are some requirements amongst these names: The location string 
must be unique amongst all bundles in the framework. The combination of 
symbolic name and version must also be unique. This means, that two (or 
more) bundles with the same name but different versions may be installed 
and active in the framework at the same time. (There is one exception: 
If the bundles symbolic name is marked as singleton, only a single 
bundle with the given symbolic name may be installed)

Given that I think multiple bundles of the same name is a special case 
and not the rule, any automatic installer should scan the bundle to be 
installed for the symbolic name and find an installed bundle with the 
same name. Then a bundle version comparison may help in deciding whether 
to upgrade, downgrade or do nothing. If no bundle is installed, 
installation is of course to take place.

BTW: The OSGi Bundle.update() method just replaces the existing bundle 
data with the new bundle data, irrespective of any symbolic name and 
version number. This allows for bundle downgrade.

> 
> For now, jcrbundles uses the last-modified value of the bundle file to
> make that decision, but it would of course be much better to get that
> info from the framework.

Hmm, I would look for a bundle with the same symbolic name and compare 
the versions: If they match, do nothing. Otherwise update the existing 
bundle or install the bundle (if no bundle with the same symbolic name 
exists).


>> ... But you raise an interesting question: What if the same bundle (same
>> symbolic name, different version) is stored in different locations ? What if
>> multiple differing configurations exist ?...
> 
> Yes, that's the problem that I mention with the LogManager config in
> SLING-587 about configs, and with bundles it's even worse: currently,
> nothing prevents you from installing the same service multiple times
> using the various front-ends, and that can lead to hard to diagnose
> problems.

I assume you are referring to "...installing the same _bundle_ multiple 
times...", right ?

> 
>>> ...Note that jcrbundles will resynchronize the OSGi state with what's in the
>>> bundles folders when the service starts: bundles and configs added, removed
>>> or updated while the service is not active are taken into account at the
>>> next restart.
>> Wow: Removed and updated ! While update may or may not work on configuration
>> files. It may not work as expected most of the time. Same for bundles.
>>
>> Consider this situation:
>>
>> * user starts jcrbundles
>> * user puts bundle A, version 1.0
>> * jcrbundles is stopped
>> * bundle A is updated to version 1.1 by some other means
>> * jcrbundles is started
>>
>> Now, you can say, we would not downgrade. That is ok here. But in some
>> situations, it is not ok (ok, generally a downgrade is not what we want)...
> 
> I haven't tested that but I think both versions of bundle A would be
> considered as distinct - at least in jcrbundles and jcrinstall, we use
> the bundle location to get it from the OSGi framework, and we prepend
> the "jcrbundles:" prefix to that location. So in this case the two
> variants of bundle A would be loaded, and 1.1 wouldn't be touched by
> jcrbundles.

Hmm, if jcrinstall would really just look for bundles with the same 
location, I would call this a bug. Correct IMHO would be to look up by 
symbolic name and compare versions.

Installing both at the same time will be problematic in most cases.

> 
> But I agree that this is a problem that must be addressed - the
> easiest short-term solution ("segregated bundles") might be to prevent
> a front-end from installing a bundle with the same Bundle-SymbolicName
> that's already been installed by another front-end.

This is IMHO not a good idea. If we would just say: "There will always 
be a single bundle with a given symbolic name", we can handle this very 
easily.

BTW: The notion of "preventing a front-end from removing a bundle, which 
was installed by another front-end is AFAICT not possible in OSGi and 
IMHO does not make any sense at all.

> 
>> ...For Remove the situation is even worse: How do you decide whether a bundle
>> or configuration has been removed if it does not exist in the repository
>> when jcrbundles starts ? I would assume, you can't....
> 
> Jcrbundles stores status nodes under /system/sling/jcrbundles/status
> for each bundle or config that it has installed, so it can find out
> that something's been deleted, and the status node gives the name of
> the bundle or config that is to be removed. That works with the
> current code.

Ah, didn't know that. Thanks for the clarification. Makes sense.

> 
>> ...So, for a startup sync, install/add is probably the only save bet we can
>> make. And not even this: Consider bundle A being removed from the framework
>> while jcrbundles has been stopped. Now jcrbundles is started again. Do you
>> really want bundle A to be reinstalled ?...
> 
> Yes, if we assume that bundles stored in the repository are
> application-level bundles that are not managed via the OSGi console,
> that would work. Implementing the segregated bundles suggestion above
> would solve that, and force people to decide if a bundle is managed by
> the OSGi console or in the repository - at least as an initial
> pragmatic solution until we have a better idea.

This is against OSGi IMHO and not even possible, because no-one (unless 
using a SecurityManager, which we don't for now) can be prevented to 
call into the framework directly. So we must be dealing with this 
situation right from the start (as we should with the Sling launcher, 
which fails in this respect for now ...)

But given that you have the status file you might assume that the bundle 
is considered installed from the POV of jcrbundles even though an 
administrator thought it would be good to remove the bundle. Removing 
the status file would cause the bundle to be reinstalled.


Regards
Felix

Re: [jira] Commented: (SLING-587) JcrBundlesManager

[Bertrand Delacretaz <bd...@apache.org>]

Hi,

On Wed, Jul 30, 2008 at 12:08 PM, Felix Meschberger <fm...@gmail.com> wrote:
> Bertrand Delacretaz (JIRA) schrieb:
>> ...Note that the same config can be edited from the Sling console, which will
>> show changes made in the cfg file, but the opposite is not true: changes
>> made in the Sling console are not visible in the cfg file...

> First off, the only correct answer to "which configuration is used" and
> "what bundle is active" comes from the running framework and not from any
> other location.

Sure.

> ...Having said that, the jcrbundles module is free to listen for configuration
> events from the ConfigurationAdmin to write back any configuration files...

I thought about that, but rewriting a config file that the user just
changed might be problematic and/or surprising, so I didn't want to
implement that in this first prototype.

> ..With bundles, it is somewhat different. Because you cannot extract the
> bundle file of installed bundles from the framework (this is probably also
> not what you want)...

Would it be possible for the OSGi framework to add for example an md5
of the bundle data as an attribute somewhere? In this way we the
various front-ends (OSGi console, jcrbundles, fileinstall, etc.) could
decide if an update is needed or not.

For now, jcrbundles uses the last-modified value of the bundle file to
make that decision, but it would of course be much better to get that
info from the framework.

> ...So, we should just keep in mind, that the jcrbundles functionality is just
> one of many frontend options to manage bundles and configurations...

100% agreed, that's how I see it.

>... But you raise an interesting question: What if the same bundle (same
> symbolic name, different version) is stored in different locations ? What if
> multiple differing configurations exist ?...

Yes, that's the problem that I mention with the LogManager config in
SLING-587 about configs, and with bundles it's even worse: currently,
nothing prevents you from installing the same service multiple times
using the various front-ends, and that can lead to hard to diagnose
problems.

>> ...Note that jcrbundles will resynchronize the OSGi state with what's in the
>> bundles folders when the service starts: bundles and configs added, removed
>> or updated while the service is not active are taken into account at the
>> next restart.
>
> Wow: Removed and updated ! While update may or may not work on configuration
> files. It may not work as expected most of the time. Same for bundles.
>
> Consider this situation:
>
> * user starts jcrbundles
> * user puts bundle A, version 1.0
> * jcrbundles is stopped
> * bundle A is updated to version 1.1 by some other means
> * jcrbundles is started
>
> Now, you can say, we would not downgrade. That is ok here. But in some
> situations, it is not ok (ok, generally a downgrade is not what we want)...

I haven't tested that but I think both versions of bundle A would be
considered as distinct - at least in jcrbundles and jcrinstall, we use
the bundle location to get it from the OSGi framework, and we prepend
the "jcrbundles:" prefix to that location. So in this case the two
variants of bundle A would be loaded, and 1.1 wouldn't be touched by
jcrbundles.

But I agree that this is a problem that must be addressed - the
easiest short-term solution ("segregated bundles") might be to prevent
a front-end from installing a bundle with the same Bundle-SymbolicName
that's already been installed by another front-end.

>
> ...For Remove the situation is even worse: How do you decide whether a bundle
> or configuration has been removed if it does not exist in the repository
> when jcrbundles starts ? I would assume, you can't....

Jcrbundles stores status nodes under /system/sling/jcrbundles/status
for each bundle or config that it has installed, so it can find out
that something's been deleted, and the status node gives the name of
the bundle or config that is to be removed. That works with the
current code.

> ...So, for a startup sync, install/add is probably the only save bet we can
> make. And not even this: Consider bundle A being removed from the framework
> while jcrbundles has been stopped. Now jcrbundles is started again. Do you
> really want bundle A to be reinstalled ?...

Yes, if we assume that bundles stored in the repository are
application-level bundles that are not managed via the OSGi console,
that would work. Implementing the segregated bundles suggestion above
would solve that, and force people to decide if a bundle is managed by
the OSGi console or in the repository - at least as an initial
pragmatic solution until we have a better idea.

> ...And of, course the security implications you also noted come to it: I assume
> the JCR location should be ACL-secured to prevent John Doe from modifying
> it. This should probably do the trick. Right ?...

Yes, we'll need that of course, and having bundles under /libs or
/apps is a first hint at what ACLs we might need.

-Bertrand

Re: [jira] Commented: (SLING-587) JcrBundlesManager

[Felix Meschberger <fm...@gmail.com>]

Hi,

Bertrand Delacretaz (JIRA) schrieb:
> Revision 680753 contains a first usable version, here's how to test it:

Good.

> Note that the same config can be edited from the Sling console, which will show changes made in the cfg file, but the opposite is not true: changes made in the Sling console are not visible in the cfg file, that's a problem that we'll need to solve. Not to mention potential security implications - we might need to restrict what can be done with jcrbundles, or make sure that it doesn't mess with bundles and configs installed via other means.

First off, the only correct answer to "which configuration is used" and 
"what bundle is active" comes from the running framework and not from 
any other location.

Having said that, the jcrbundles module is free to listen for 
configuration events from the ConfigurationAdmin to write back any 
configuration files.

With bundles, it is somewhat different. Because you cannot extract the 
bundle file of installed bundles from the framework (this is probably 
also not what you want).

So, we should just keep in mind, that the jcrbundles functionality is 
just one of many frontend options to manage bundles and configurations.

But you raise an interesting question: What if the same bundle (same 
symbolic name, different version) is stored in different locations ? 
What if multiple differing configurations exist ?

> Note that jcrbundles will resynchronize the OSGi state with what's in the bundles folders when the service starts: bundles and configs added, removed or updated while the service is not active are taken into account at the next restart.

Wow: Removed and updated ! While update may or may not work on 
configuration files. It may not work as expected most of the time. Same 
for bundles.

Consider this situation:

* user starts jcrbundles
* user puts bundle A, version 1.0
* jcrbundles is stopped
* bundle A is updated to version 1.1 by some other means
* jcrbundles is started

Now, you can say, we would not downgrade. That is ok here. But in some 
situations, it is not ok (ok, generally a downgrade is not what we want)

For Remove the situation is even worse: How do you decide whether a 
bundle or configuration has been removed if it does not exist in the 
repository when jcrbundles starts ? I would assume, you can't.

So, for a startup sync, install/add is probably the only save bet we can 
make. And not even this: Consider bundle A being removed from the 
framework while jcrbundles has been stopped. Now jcrbundles is started 
again. Do you really want bundle A to be reinstalled ?

And of, course the security implications you also noted come to it: I 
assume the JCR location should be ACL-secured to prevent John Doe from 
modifying it. This should probably do the trick. Right ?


Regards
Felix