Re: OSGi support round two

[Ittay Dror <it...@gmail.com>]

Assaf Arkin wrote:
> 
> I circulated a proposal on buildr-dev to make transitive dependencies and
> version matching the focal point for Buildr 1.4.  Briefly it will allow
> you
> to do something like this:
> 
>   compile.with foo
> 
> Which will download, install and compile with foo, but also all of foo's
> dependencies.  Or if foo is another project, bring all of that project's
> dependencies with it.  It will help keep buildfiles smaller by only
> specifying the top packages it's using, and let Buildr figure out the
> rest.
> 
For Java, I don't think this is a good choice (or, at least not good if this
is done the maven way). Say foo uses log4j. Naturally it needs this to
compile. But the project that uses foo doesn't. First thing is that log4j is
downloaded for no reason. It is also added to the javac classpath, but not
much harm here. Now, the project using foo starts using log4j. It compiles
just fine until the foo project decides to change the logging mechanism it
uses. Now the project using it is broken. Also, this can cause jar hell - if
my project uses one version of log4j and foo uses another (or my project
uses foo and bar and they each use a different version of log4j). The best
is to separate dependencies so that foo defines dependencies for it to
compile and a subset of those for clients of it to compile. I think the OSGi
way will have the same problems. Why not, use Ivy (programmatically of
course)?

Ittay


Assaf Arkin wrote:
> 
> Version matching means that foo can specify the package name but no
> version
> number, and Buildr will attempt to match the most recent version for that
> package.  Or you can specify some constraints, like >= 1.2 or != 1.3.
> 
> 
> Right now we provide support for using and publishing packages, and it's
> loosely based on the Maven 2 repository model.  We got the basics working,
> just no support for transitive dependencies (other than an experimental
> transitive method) and version matching.  To that we can also add OSGi
> support, which means putting OSGi meta-data inside the packages (in
> MANIFEST.MF) and reading it from them, in addition or instead of POM
> (depending on source and target).
> 
> The problem is, doing both Maven and OSGi, and transitive dependencies and
> version matching.  They don't like to co-exist, because OSGi and Maven
> handle dependencies and version numbers differently.  Without going into
> too
> much details, I'll illustrate with a simple example.
> 
> Let's say you have three versions of the same JAR: 1.2.0-RC2, 1.2.0-2 and
> 1.2.0-10.  The way Maven works, if the dependency just specifies "1.2",
> then
> the most recent version is 1.2.0-10 because 10 is higher than 2 and
> numbers
> are higher than alphanumeric.
> 
> The way OSGi (and most other packaging mechanisms) work, the last part of
> the version number (qualifier) is just a string.  Being a string, the most
> recent version is RC2, next is 2, and 10 is the oldest.  Developers who
> use
> OSGi never run into this problem because they package everything using
> versions like 1.2.0, 1.2.0-B45 or 1.2.0-R20080506.
> 
> 
> If we pick one model and support it natively, it will get all the cool
> features and we'll do best effort on the other model.  Say we pick OSGi,
> then we'll have a local repository that looks like an OSGi repository,
> we'll
> use OSGi version numbers, and make Maven artifacts look like OSGi bundles.
>  In reverse, if we go with Maven, we keep the same local repository model,
> etc
> 
> Some things I'm not sure we can even do, e.g. Maven snapshots mess with
> OSGi, not sure we can support them if we do both, but OSGi has timestamped
> version numbers that achieve the same thing.
> 
> So part two is which one of these two models we do natively for transitive
> dependencies and version matching, and which one we fake to work like the
> other?
> 
> Assaf
> 
> 

-- 
View this message in context: http://www.nabble.com/OSGi-support-round-two-tp17118555p18108562.html
Sent from the Buildr - Dev mailing list archive at Nabble.com.

Re: OSGi support round two

[Assaf Arkin <ar...@intalio.com>]

On Wed, Jun 25, 2008 at 2:46 AM, Ittay Dror <it...@gmail.com> wrote:
>
>
>
> Assaf Arkin wrote:
>>
>> I circulated a proposal on buildr-dev to make transitive dependencies and
>> version matching the focal point for Buildr 1.4.  Briefly it will allow
>> you
>> to do something like this:
>>
>>   compile.with foo
>>
>> Which will download, install and compile with foo, but also all of foo's
>> dependencies.  Or if foo is another project, bring all of that project's
>> dependencies with it.  It will help keep buildfiles smaller by only
>> specifying the top packages it's using, and let Buildr figure out the
>> rest.
>>
> For Java, I don't think this is a good choice (or, at least not good if this
> is done the maven way). Say foo uses log4j. Naturally it needs this to
> compile. But the project that uses foo doesn't.

OSGi generally lists runtime dependencies.  So if foo needs log4j to
compile, and only to compile, it shouldn't show up in the dependency
list as a runtime dependency.  You're right that there's no reason for
Buildr to download such a dependency, it just adds something else that
might break the build.

On the other hand, if foo needs log4j to run, then once you start
running foo, it will break without log4j being there.  You won't be
able to test the project or deploy it without having that dependency.
In which case I think it's a good idea for the project not to build
without it.

> First thing is that log4j is
> downloaded for no reason. It is also added to the javac classpath, but not
> much harm here. Now, the project using foo starts using log4j. It compiles
> just fine until the foo project decides to change the logging mechanism it
> uses. Now the project using it is broken. Also, this can cause jar hell - if
> my project uses one version of log4j and foo uses another (or my project
> uses foo and bar and they each use a different version of log4j).

What would happen at runtime?

I'm looking at this from the perspective of runtime.  It's the end
goal of the buildfile, to create something you can deploy and use, and
I think it should be the most important goal of the dependency
management solution we pick.  Making the buildfile smaller and the
build process easier are also damn important, but secondary goal to
repeatedly getting an image you can deploy and execute.

So if foo and bar can't agree to run with the same version of log4j,
there's just no way you can deploy and execute those.  Even if you can
compile them, and you'll be able to because Buildr will give you the
flexibility to ignore dependencies, they still won't run.

If bar stops caring for log4j because it doesn't use it anymore, but
foo still needs it to run, then you want log4j to stick around, you
want the dependency management to include that dependency.  So that's
a good feature dependency management brings.

If foo can use any version of log4j, but bar knows better and needs
the latest one (maybe it logs a lot and the latest version is faster),
we need a way to pick the more precise dependency, or allow you to fix
foo's dependency so they can be run together.


> The best
> is to separate dependencies so that foo defines dependencies for it to
> compile and a subset of those for clients of it to compile.

We definitely need that flexibility.  There are libraries you will
only use during compiling (the JDK runtime library is one of those, so
is anything you need only for processing annotations), others only
required during runtime.  So there's going to be a difference between
compile, runtime and test dependencies.

> I think the OSGi
> way will have the same problems. Why not, use Ivy (programmatically of
> course)?

Once you get the project to build, what happens next?  In many cases
the answer is not much, you're producing a library JAR with no
dependencies, or maybe a full application packages as a Zip.  But in
other cases the next step is deploying to a runtime environment that
needs to have the right set of dependencies to allow your code to
execute properly.  So now there's a lot more that needs to be done
after the build to get the right combination of pieces deployed.

OSGi manifests allow you to exert control over that, you can create a
package with dependency information that the runtime environment can
use.  Of course, it only works if your runtime supports OSGi
dependencies.  I'm seeing enough momentum behind that, that I think
it's worth considering.

Assaf

>
> Ittay
>
>
> Assaf Arkin wrote:
>>
>> Version matching means that foo can specify the package name but no
>> version
>> number, and Buildr will attempt to match the most recent version for that
>> package.  Or you can specify some constraints, like >= 1.2 or != 1.3.
>>
>>
>> Right now we provide support for using and publishing packages, and it's
>> loosely based on the Maven 2 repository model.  We got the basics working,
>> just no support for transitive dependencies (other than an experimental
>> transitive method) and version matching.  To that we can also add OSGi
>> support, which means putting OSGi meta-data inside the packages (in
>> MANIFEST.MF) and reading it from them, in addition or instead of POM
>> (depending on source and target).
>>
>> The problem is, doing both Maven and OSGi, and transitive dependencies and
>> version matching.  They don't like to co-exist, because OSGi and Maven
>> handle dependencies and version numbers differently.  Without going into
>> too
>> much details, I'll illustrate with a simple example.
>>
>> Let's say you have three versions of the same JAR: 1.2.0-RC2, 1.2.0-2 and
>> 1.2.0-10.  The way Maven works, if the dependency just specifies "1.2",
>> then
>> the most recent version is 1.2.0-10 because 10 is higher than 2 and
>> numbers
>> are higher than alphanumeric.
>>
>> The way OSGi (and most other packaging mechanisms) work, the last part of
>> the version number (qualifier) is just a string.  Being a string, the most
>> recent version is RC2, next is 2, and 10 is the oldest.  Developers who
>> use
>> OSGi never run into this problem because they package everything using
>> versions like 1.2.0, 1.2.0-B45 or 1.2.0-R20080506.
>>
>>
>> If we pick one model and support it natively, it will get all the cool
>> features and we'll do best effort on the other model.  Say we pick OSGi,
>> then we'll have a local repository that looks like an OSGi repository,
>> we'll
>> use OSGi version numbers, and make Maven artifacts look like OSGi bundles.
>>  In reverse, if we go with Maven, we keep the same local repository model,
>> etc
>>
>> Some things I'm not sure we can even do, e.g. Maven snapshots mess with
>> OSGi, not sure we can support them if we do both, but OSGi has timestamped
>> version numbers that achieve the same thing.
>>
>> So part two is which one of these two models we do natively for transitive
>> dependencies and version matching, and which one we fake to work like the
>> other?
>>
>> Assaf
>>
>>
>
> --
> View this message in context: http://www.nabble.com/OSGi-support-round-two-tp17118555p18108562.html
> Sent from the Buildr - Dev mailing list archive at Nabble.com.
>
>