You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@felix.apache.org by David Jencks <da...@yahoo.com> on 2013/10/15 19:51:25 UTC

Why DependencyManager rather than DS?

After seeing a lot of commit activity on DependencyManager I decided to try to understand what it's for, and after looking at the documentation I'm still not sure.  It looks to me like the main feature is a fluent api that provides something like DS, although less declaratively, and then there are some special cases that might be slightly simpler than just declaring a service that does the same thing (aspects, temporal)

So as a DS partisan :-) I'm wondering what the big advantages of DependencyManager are.

I also wonder if it would be useful to add to DS a fluent api for adding components at runtime.  I think this would be pretty easy, just construct ComponentMetadata and add it to the appropriate BundleComponentActivator.

thanks
david jencks

Re: Why DependencyManager rather than DS?

Posted by "Richard S. Hall" <he...@ungoverned.org>.
On 10/15/13 13:51 , David Jencks wrote:
> After seeing a lot of commit activity on DependencyManager I decided to try to understand what it's for, and after looking at the documentation I'm still not sure.  It looks to me like the main feature is a fluent api that provides something like DS, although less declaratively, and then there are some special cases that might be slightly simpler than just declaring a service that does the same thing (aspects, temporal)
>
> So as a DS partisan :-) I'm wondering what the big advantages of DependencyManager are.

I'll let Marcel et al answer this portion of your question.

> I also wonder if it would be useful to add to DS a fluent api for adding components at runtime.  I think this would be pretty easy, just construct ComponentMetadata and add it to the appropriate BundleComponentActivator.

Actually, because of DM the idea of spec'ing something like its API for 
DS had been discussed inside the OSGi Alliance a while back, not sure 
why it didn't go anywhere.

-> richard

> thanks
> david jencks
>

Re: Why DependencyManager rather than DS?

Posted by Marcel Offermans <ma...@luminis.nl>.
Hello David,

Sit down first, this is a long mail. :)

On Oct 16, 2013, at 1:06 AM, David Jencks <da...@yahoo.com> wrote:
> On Oct 15, 2013, at 1:33 PM, Marcel Offermans <ma...@luminis.nl> wrote:
>> On Oct 15, 2013, at 19:51 PM, David Jencks <da...@yahoo.com> wrote:

>> DS still can't do dynamic dependencies, nor is it extensible to support new types of dependencies (DM comes with dependencies to track services, configuration, bundles and "resources". To give an example, DM can declare a component that requires service A and configuration B. As soon as it has both, the component can evaluate configuration B and depending on its contents add another service dependency C (or something like that).
> 
> Do you have an example?  The "requires service A and configuration B" sounds like in DS a 1..1 reference to A and ConfigurationPolicy.REQUIRE.

This part is the same as DS, and it would look somewhat like this in DM:

public class Activator extends DependencyActivatorBase {
	public void init(BundleContext c, DependencyManager dm) throws Exception {
		dm.add(createComponent()
			.setImplementation(ComponentWithDynamicDependencies.class)
			.add(createServiceDependency()
				.setService(EventAdmin.class)
				.setRequired(true))
			.add(createConfigurationDependency()
				.setPid("org.foo.pid")));
	}

	public void destroy(BundleContext c, DependencyManager dm) throws Exception {
	}
}

So we end up with a component that will be instantiated lazily by creating an instance of ComponentWithDynamicDependencies as soon as the EventAdmin service is available and a configuration called "org.foo.pid". For that configuration we can also do very specific checks in code to determine if it contains all the information we want. If not, DM will not consider the requirement as "satisfied" and the component will wait until a good configuration comes along (see updated method below).

> What adds the dependency on C?  How does this differ from including the C.target property in the configuration?

The component does, programmatically, as soon as the two requirements above have been satisfied. Let's continue with our example:

public class ComponentWithDynamicDependencies implements ManagedService {
	private volatile EventAdmin m_eventAdmin;
	private String m_filter;
	private boolean m_isRequired;
	
	public void init(Component c) {
		DependencyManager dm = c.getDependencyManager();
		c.add(dm.createServiceDependency()
			.setService(Storage.class, m_filter)
			.setRequired(m_isRequired)
			.setInstanceBound(true));
	}

	public void updated(Dictionary properties) throws ConfigurationException {
		m_filter = (String) getNotNull(properties, "filter");
		m_isRequired = (Boolean) getNotNull(properties, "required");
	}
	
	private Object getNotNull(Dictionary properties, String key) throws ConfigurationException {
		Object result = properties.get(key);
		if (result == null) {
			throw new ConfigurationException(key, "must be specified");
		}
		return result;
	}
}

So the "updated" method checks the incoming properties and checks if the two ones we need are really there. Here you could (and maybe should) do more consistency checks, such as make sure you got a valid filter, etc. but I left that out to keep the example simple).

Then, the "init" method is automatically invoked as part of the life cycle of the component, as soon as the two dependencies are present. It takes the "Component", as previously declared in the Activator, as an optional argument and in this case we use that to add an extra service dependency on a "Storage" class with the filter condition that was in the configuration data. Also, a second condition determines if the dependency is optional or required.

>> DM also has concepts like aspects and adapters, that allow you to declare factories that automatically generate components that attach to other services. In the case of aspects creating a whole "chain" of services, allowing you to easily intercept existing services and add behaviour such as adding a cache to a store. In the case of adapters allowing you to add for example management capabilities to services.. Just to name a few examples. This really deserves a longer description, but this gives you a general idea.
> 
> From my quick look at the docs it looked like these examples basically registered another service exposing the same interface and with a dependency on the original service, something that is easy to do with DS as well.  I didn't see the factory aspect described….. again, is there a more sophisticated example?

Ok, let's take an example using the whiteboard pattern to create a component that can execute all Task services that have been published in the framework. For the sake of simplicity I'm going to publish that component and two Task implementations from the same bundle, but in the real world you'd probably create different bundles for them:

public class Activator extends DependencyActivatorBase {
	public void init(BundleContext c, DependencyManager dm) throws Exception {
		dm.add(createComponent()
			.setImplementation(TaskExecutor.class)
			.add(createServiceDependency()
				.setService(Task.class)
				.setCallbacks("add", "remove")));
		
		dm.add(createComponent()
			.setInterface(Task.class.getName(), null)
			.setImplementation(TaskOne.class));
		
		dm.add(createComponent()
			.setInterface(Task.class.getName(), null)
			.setImplementation(TaskTwo.class));

We'll continue the activator a bit further down. The other classes used are:

public class TaskExecutor {
	public void add(Task task) {
		// store our task for later execution
	}
	
	public void remove(Task task) {
		// remove our task from the internal list
	}
}

public interface Task {
	public void execute();
}

public class TaskOne implements Task {
	public void execute() {
		System.out.println("Task One");
	}
}

public class TaskTwo implements Task {
	public void execute() {
		System.out.println("Task Two");
	}
}

Nothing "special" so far. Two Task services come up and are "collected" by the TaskExecutor.

Now let's add an aspect. Aspects are factories that create another service on top of the service that they're an aspect for. In this example, let's create an aspect on top of Task services that counts the number of invocations for that specific task. We continue our activator like this:

		dm.add(createAspectService(Task.class, null, 10)
			.setImplementation(TaskCounter.class));

So this creates two instances of a Task service that bind to each underlying Task (TaskOne and TaskTwo). DM automatically injects the original service in the aspect, and in this case, the aspect is created with a service ranking of 10. You can create multiple aspects and create a whole chain if you want. The order is determined by the ranking you supply.

DM also takes care that the TaskExecutor will now see only the two aspects on top of the original tasks (in other words, it will always only see the top of the chain).

The implementation of the aspect is something like this:

public class TaskCounter implements Task {
	private volatile Task m_task;
	private long m_counter = 0;

	public void execute() {
		m_counter++;
		m_task.execute();
		System.out.println("Invoked " + m_task + " " + m_counter + " times.");
	}
}

Now, let's also add an adapter. Adapters also bind to services, but don't create a "chain" like aspects do. Assume we would like to create a servlet for each task so we can trigger it with an HTTP GET call to that servlet. Going back to the activator we declare our adapter, making sure it also has a service dependency on HttpService:

		dm.add(createAdapterService(Task.class, null)
			.setImplementation(TaskServlet.class)
			.add(createServiceDependency()
				.setService(HttpService.class)
				.setRequired(true)));

What happens now is that we will end up with an instance of TaskServlet that is linked to the Task and that also has HttpService injected. So in that implementation, we can simply do our magic in the start() method (which is one of the life cycle methods that will be invoked on a component, if present):

public class TaskServlet {
	private volatile HttpService m_service;
	private volatile Task m_task;
	
	public void start() {
		// register as a servlet here and invoke the task from its HTTP GET method
	}
}

I did not actually include the code to register the servlet, but you can see where it would go.

There are still a few other features that I did not discuss in this mail, but aspects and adapters have proven to be very valuable when building modular systems to allow you to really build components that do one thing well. Just as an other example, by adding aspects to certain services, it's easy to deploy all kinds of monitoring capabilities into an existing application, without ever having to touch the original code. Just deploy an extra bundle with an aspect, and you're done.

>> A third feature that might be interesting is that DM also has support for "indices" that dramatically speed up the OSGi service registry when you're dealing with applications that consist of lots of services (1k-10k or even more). It uses a technique similar to what relational databases use to speed up lookups. Xander did a lot of work on this, they have a huge application that used to take about 30 minutes to start up and now does so in more like 30 seconds (so orders of magnitude speedup).
> 
> I'd like to look at this, can you point me to the code?

Sure, there is a static {} initializer block in:

dependencymanager/core/src/main/java/org/apache/felix/dm/DependencyManager.java

That sets up the indices based on a system property that needs to be set. That currently means that, like with databases, you need to think about what indices you want to use to speed up your application, which obviously depends on the types of services and dependencies that you have. If you then look at the folder:

dependencymanager/core/src/main/java/org/apache/felix/dm/impl/index

You will see the indices for which we currently have implementations. Most of those can be configured. Some are for specific scenarios (such as when you use a lot of adapters or aspects as mentioned above).

Mind you, it's probably not the easiest piece of code to read.

>>> I also wonder if it would be useful to add to DS a fluent api for adding components at runtime.  I think this would be pretty easy, just construct ComponentMetadata and add it to the appropriate BundleComponentActivator.
>> 
>> Creating the fluent API would not be too hard, but DS is not dynamic, you need to package the XML with the bundle for it to work, so that part would be harder to fix (unless you resort to generating bundles on the fly or something like that).
> 
> I was thinking of creating a non-xml way to add the same kind of information as in the xml, I think you might describe this as making DS dynamic.

So how would you do that at build time? Build the code that uses the fluent API, run it and have it output XML that can then be put in the bundle? That could definitely work.

Greetings, Marcel

Re: Why DependencyManager rather than DS?

Posted by David Jencks <da...@yahoo.com>.
Hi Marcel, thanks for the explanations…. more quibbling inline :-)

On Oct 15, 2013, at 1:33 PM, Marcel Offermans <ma...@luminis.nl> wrote:

> Hello David,
> 
> On Oct 15, 2013, at 19:51 PM, David Jencks <da...@yahoo.com> wrote:
> 
>> After seeing a lot of commit activity on DependencyManager I decided to try to understand what it's for, and after looking at the documentation I'm still not sure.  It looks to me like the main feature is a fluent api that provides something like DS, although less declaratively, and then there are some special cases that might be slightly simpler than just declaring a service that does the same thing (aspects, temporal)
> 
> DependencyManager has its roots long time ago, when there was no Declarative Services specification yet. It started when I was working on my first big OSGi project (about 10 years ago) and we needed a library to help us declare and manage dependencies.
> 
> The only library at that time was "servicebinder", which was somewhat similar to what became DS. We evaluated that library for our project, but it did not fulfill all our use cases.
> 
> Most importantly, we had use cases that required us to declare dependencies at runtime, for example based on configuration data or some hardware aspects we discovered at runtime. Furthermore, I did not like the XML configuration, which did not automatically update when refactoring your code and did not have code completion or syntax checking.
> 
> That last bit has been improved now that DS supports annotations to generate XML.
> 
>> So as a DS partisan :-) I'm wondering what the big advantages of DependencyManager are.
> 
> DS still can't do dynamic dependencies, nor is it extensible to support new types of dependencies (DM comes with dependencies to track services, configuration, bundles and "resources". To give an example, DM can declare a component that requires service A and configuration B. As soon as it has both, the component can evaluate configuration B and depending on its contents add another service dependency C (or something like that).

Do you have an example?  The "requires service A and configuration B" sounds like in DS a 1..1 reference to A and ConfigurationPolicy.REQUIRE.  What adds the dependency on C?  How does this differ from including the C.target property in the configuration?

> 
> DM also has concepts like aspects and adapters, that allow you to declare factories that automatically generate components that attach to other services. In the case of aspects creating a whole "chain" of services, allowing you to easily intercept existing services and add behaviour such as adding a cache to a store. In the case of adapters allowing you to add for example management capabilities to services.. Just to name a few examples. This really deserves a longer description, but this gives you a general idea.

From my quick look at the docs it looked like these examples basically registered another service exposing the same interface and with a dependency on the original service, something that is easy to do with DS as well.  I didn't see the factory aspect described….. again, is there a more sophisticated example?

> 
> A third feature that might be interesting is that DM also has support for "indices" that dramatically speed up the OSGi service registry when you're dealing with applications that consist of lots of services (1k-10k or even more). It uses a technique similar to what relational databases use to speed up lookups. Xander did a lot of work on this, they have a huge application that used to take about 30 minutes to start up and now does so in more like 30 seconds (so orders of magnitude speedup).

I'd like to look at this, can you point me to the code?

> 
>> I also wonder if it would be useful to add to DS a fluent api for adding components at runtime.  I think this would be pretty easy, just construct ComponentMetadata and add it to the appropriate BundleComponentActivator.
> 
> Creating the fluent API would not be too hard, but DS is not dynamic, you need to package the XML with the bundle for it to work, so that part would be harder to fix (unless you resort to generating bundles on the fly or something like that).

I was thinking of creating a non-xml way to add the same kind of information as in the xml, I think you might describe this as making DS dynamic.

thanks!
david jencks


> 
> The other way round would be easier: creating an extender bundle that reads the XML descriptors that DS uses and using DM to create the appropriate components.
> 
> For the record, DM currently also has an annotation based API, contributed a while ago by Pierre and Arjun.
> 
> Greetings, Marcel
>

Re: Why DependencyManager rather than DS?

Posted by Marcel Offermans <ma...@luminis.nl>.
Hello David,

On Oct 15, 2013, at 19:51 PM, David Jencks <da...@yahoo.com> wrote:

> After seeing a lot of commit activity on DependencyManager I decided to try to understand what it's for, and after looking at the documentation I'm still not sure.  It looks to me like the main feature is a fluent api that provides something like DS, although less declaratively, and then there are some special cases that might be slightly simpler than just declaring a service that does the same thing (aspects, temporal)

DependencyManager has its roots long time ago, when there was no Declarative Services specification yet. It started when I was working on my first big OSGi project (about 10 years ago) and we needed a library to help us declare and manage dependencies.

The only library at that time was "servicebinder", which was somewhat similar to what became DS. We evaluated that library for our project, but it did not fulfill all our use cases.

Most importantly, we had use cases that required us to declare dependencies at runtime, for example based on configuration data or some hardware aspects we discovered at runtime. Furthermore, I did not like the XML configuration, which did not automatically update when refactoring your code and did not have code completion or syntax checking.

That last bit has been improved now that DS supports annotations to generate XML.

> So as a DS partisan :-) I'm wondering what the big advantages of DependencyManager are.

DS still can't do dynamic dependencies, nor is it extensible to support new types of dependencies (DM comes with dependencies to track services, configuration, bundles and "resources". To give an example, DM can declare a component that requires service A and configuration B. As soon as it has both, the component can evaluate configuration B and depending on its contents add another service dependency C (or something like that).

DM also has concepts like aspects and adapters, that allow you to declare factories that automatically generate components that attach to other services. In the case of aspects creating a whole "chain" of services, allowing you to easily intercept existing services and add behaviour such as adding a cache to a store. In the case of adapters allowing you to add for example management capabilities to services.. Just to name a few examples. This really deserves a longer description, but this gives you a general idea.

A third feature that might be interesting is that DM also has support for "indices" that dramatically speed up the OSGi service registry when you're dealing with applications that consist of lots of services (1k-10k or even more). It uses a technique similar to what relational databases use to speed up lookups. Xander did a lot of work on this, they have a huge application that used to take about 30 minutes to start up and now does so in more like 30 seconds (so orders of magnitude speedup).

> I also wonder if it would be useful to add to DS a fluent api for adding components at runtime.  I think this would be pretty easy, just construct ComponentMetadata and add it to the appropriate BundleComponentActivator.

Creating the fluent API would not be too hard, but DS is not dynamic, you need to package the XML with the bundle for it to work, so that part would be harder to fix (unless you resort to generating bundles on the fly or something like that).

The other way round would be easier: creating an extender bundle that reads the XML descriptors that DS uses and using DM to create the appropriate components.

For the record, DM currently also has an annotation based API, contributed a while ago by Pierre and Arjun.

Greetings, Marcel