You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Nathan Beyer <nd...@apache.org> on 2009/04/30 02:50:30 UTC
xerces and xalan dependencies
I've been thinking about how we consume Xerces and Xalan, especially
since we've had to do some of the more recent modifications to the
build scripts to manipulate the JARs and archives in various ways. As
an alternative to grabbing the binary packages, we could grab the
source code itself and do our own builds. We could do this by grabbing
the officially distributed source archives or we could checkout the
code directly via svn:externals pointing to release tags (some risk in
that).
One advantage this has is that the code would be compiled at the
bytecode level of our code (Xerces currently builds to support Java
1.3). The other would be a more natural fit into the classlib module
structure, which would allow us to build and package manifests as well
as additional tests.
Just something I've been knocking around in my head. Any comments or
additional thoughts?
Re: xerces and xalan dependencies
Posted by Tony Wu <wu...@gmail.com>.
I think it's a general idea which applies to all the dependencies such
as ICU, BouncyCastle, etc.
We need to understand one thing that is comparing to the side effect
of folking the src code, what benefits we can harvest. another is
whether they are allowed to be redistributed.
It should be disscussed case by case,
XML related
you have mentioned that we can
1. rebuild them to the same bytecode level
2. make them more natural as harmony modules.
ICU
It's the one which I want to customize as soon as I can. With ability
to change the src code and build, we can
1. remove the unnecessary cycle dependency from ICU code rather that a
work around in harmony code.
2. Improve performance - this area is one of the worst against RI.
3. remove unnecessary data and duplicate charsets to reduce the
disk/memory footprint
BouncyCastle:
A quick test shows that it consumes more memories than RI - still dont
know why. but changes to the src code is definitely required to fix
that. Also we can do customization to include basic algorithms into
the core of Harmony Select which are just talking about to reduce the
size and achieve better modulization.
YOKO
Anyone want to do this?
BCEL
do we still need that? I see ASM has been included.
just to stimulate the discussion, need more thoughts.
On Thu, Apr 30, 2009 at 8:50 AM, Nathan Beyer <nd...@apache.org> wrote:
> I've been thinking about how we consume Xerces and Xalan, especially
> since we've had to do some of the more recent modifications to the
> build scripts to manipulate the JARs and archives in various ways. As
> an alternative to grabbing the binary packages, we could grab the
> source code itself and do our own builds. We could do this by grabbing
> the officially distributed source archives or we could checkout the
> code directly via svn:externals pointing to release tags (some risk in
> that).
>
> One advantage this has is that the code would be compiled at the
> bytecode level of our code (Xerces currently builds to support Java
> 1.3). The other would be a more natural fit into the classlib module
> structure, which would allow us to build and package manifests as well
> as additional tests.
>
> Just something I've been knocking around in my head. Any comments or
> additional thoughts?
>
--
Tony Wu
China Software Development Lab, IBM
Re: xerces and xalan dependencies
Posted by Nathan Beyer <nd...@apache.org>.
On Thu, Apr 30, 2009 at 3:19 AM, Tim Ellison <t....@gmail.com> wrote:
> Tony Wu wrote:
>> On Thu, Apr 30, 2009 at 3:19 PM, Mark Hindess
>> <ma...@googlemail.com> wrote:
>>> In message <3b...@mail.gmail.com>,
>>> Nathan Beyer writes:
>>>> I've been thinking about how we consume Xerces and Xalan, especially
>>>> since we've had to do some of the more recent modifications to the
>>>> build scripts to manipulate the JARs and archives in various ways. As
>>>> an alternative to grabbing the binary packages, we could grab the
>>>> source code itself and do our own builds. We could do this by grabbing
>>>> the officially distributed source archives or we could checkout the
>>>> code directly via svn:externals pointing to release tags (some risk in
>>>> that).
>>> I'd rather not use svn:externals.
>>
>> I dont like svn:externals either. Probably it's not necessary to keep
>> them up to date. How about vote for whether to update when discussing
>> the next milestone build. I incline to grab the src code so that we
>> won't heavily depends on the script.
>
> +1 to not using svn:externals. It plays havoc with our version control!
>
>>>> One advantage this has is that the code would be compiled at the
>>>> bytecode level of our code (Xerces currently builds to support Java
>>>> 1.3). The other would be a more natural fit into the classlib module
>>>> structure, which would allow us to build and package manifests as well
>>>> as additional tests.
>>> I've been thinking about creating modules/xml (and modules/orb) so that
>>> these dependencies are handled in a way that is more consistent with
>>> our modular structure. It would allow us to do things like
>>> -Dexclude.module=orb and not have to download the yoko dependency (which
>>> would be sensible for Harmony Select).
>>>
>> yes, I think it is the consistent with Nathan's "more natural fit into
>> the classlib module structure"
>
> yep
>
>>>> Just something I've been knocking around in my head. Any comments or
>>>> additional thoughts?
>>> We don't really do very much to the jar. (Now that we are doing it only
>>> once per download rather than once per build) I don't think it is a big
>>> deal. However, I like the idea of actually running xerces and xalan
>>> tests on Harmony and perhaps the best way to do that is to grab source -
>>> unless the tests are in a jar?
>>
>> good point. we'd better run these tests on Harmony.
>
> All good points. The only minor concern I have is that is we start to
> modify the dependencies (too much) we'll get into a world of hurt when
> trying to upgrade to the latest versions.
>
> Rather than locally customize a particular distribution there should be
> a point where we work closer with the dependency project to produce
> something more closely aligned to our use case.
Just to clarify one point - I am NOT talking about forking the code at
all. I was thinking more of a svn:externals, which I know people don't
like, a classlib-local type of "populate-src" or grabbing the source
archives, like we do for APR. I'm open to this type of dependency grab
for other Java dependencies too.
I've played in the Xerces code base from time to time and it's
extremely stable, but there are very few tests. I'm not as familiar
with the Xalan code base.
-Nathan
Re: xerces and xalan dependencies
Posted by Tim Ellison <t....@gmail.com>.
Tony Wu wrote:
> On Thu, Apr 30, 2009 at 3:19 PM, Mark Hindess
> <ma...@googlemail.com> wrote:
>> In message <3b...@mail.gmail.com>,
>> Nathan Beyer writes:
>>> I've been thinking about how we consume Xerces and Xalan, especially
>>> since we've had to do some of the more recent modifications to the
>>> build scripts to manipulate the JARs and archives in various ways. As
>>> an alternative to grabbing the binary packages, we could grab the
>>> source code itself and do our own builds. We could do this by grabbing
>>> the officially distributed source archives or we could checkout the
>>> code directly via svn:externals pointing to release tags (some risk in
>>> that).
>> I'd rather not use svn:externals.
>
> I dont like svn:externals either. Probably it's not necessary to keep
> them up to date. How about vote for whether to update when discussing
> the next milestone build. I incline to grab the src code so that we
> won't heavily depends on the script.
+1 to not using svn:externals. It plays havoc with our version control!
>>> One advantage this has is that the code would be compiled at the
>>> bytecode level of our code (Xerces currently builds to support Java
>>> 1.3). The other would be a more natural fit into the classlib module
>>> structure, which would allow us to build and package manifests as well
>>> as additional tests.
>> I've been thinking about creating modules/xml (and modules/orb) so that
>> these dependencies are handled in a way that is more consistent with
>> our modular structure. It would allow us to do things like
>> -Dexclude.module=orb and not have to download the yoko dependency (which
>> would be sensible for Harmony Select).
>>
> yes, I think it is the consistent with Nathan's "more natural fit into
> the classlib module structure"
yep
>>> Just something I've been knocking around in my head. Any comments or
>>> additional thoughts?
>> We don't really do very much to the jar. (Now that we are doing it only
>> once per download rather than once per build) I don't think it is a big
>> deal. However, I like the idea of actually running xerces and xalan
>> tests on Harmony and perhaps the best way to do that is to grab source -
>> unless the tests are in a jar?
>
> good point. we'd better run these tests on Harmony.
All good points. The only minor concern I have is that is we start to
modify the dependencies (too much) we'll get into a world of hurt when
trying to upgrade to the latest versions.
Rather than locally customize a particular distribution there should be
a point where we work closer with the dependency project to produce
something more closely aligned to our use case.
Regards,
Tim
Re: xerces and xalan dependencies
Posted by Tony Wu <wu...@gmail.com>.
On Thu, Apr 30, 2009 at 3:19 PM, Mark Hindess
<ma...@googlemail.com> wrote:
>
> In message <3b...@mail.gmail.com>,
> Nathan Beyer writes:
>>
>> I've been thinking about how we consume Xerces and Xalan, especially
>> since we've had to do some of the more recent modifications to the
>> build scripts to manipulate the JARs and archives in various ways. As
>> an alternative to grabbing the binary packages, we could grab the
>> source code itself and do our own builds. We could do this by grabbing
>> the officially distributed source archives or we could checkout the
>> code directly via svn:externals pointing to release tags (some risk in
>> that).
>
> I'd rather not use svn:externals.
I dont like svn:externals either. Probably it's not necessary to keep
them up to date. How about vote for whether to update when discussing
the next milestone build. I incline to grab the src code so that we
won't heavily depends on the script.
>
>> One advantage this has is that the code would be compiled at the
>> bytecode level of our code (Xerces currently builds to support Java
>> 1.3). The other would be a more natural fit into the classlib module
>> structure, which would allow us to build and package manifests as well
>> as additional tests.
>
> I've been thinking about creating modules/xml (and modules/orb) so that
> these dependencies are handled in a way that is more consistent with
> our modular structure. It would allow us to do things like
> -Dexclude.module=orb and not have to download the yoko dependency (which
> would be sensible for Harmony Select).
>
yes, I think it is the consistent with Nathan's "more natural fit into
the classlib module structure"
>> Just something I've been knocking around in my head. Any comments or
>> additional thoughts?
>
> We don't really do very much to the jar. (Now that we are doing it only
> once per download rather than once per build) I don't think it is a big
> deal. However, I like the idea of actually running xerces and xalan
> tests on Harmony and perhaps the best way to do that is to grab source -
> unless the tests are in a jar?
good point. we'd better run these tests on Harmony.
>
> Regards,
> Mark.
>
>
>
--
Tony Wu
China Software Development Lab, IBM
Re: xerces and xalan dependencies
Posted by Mark Hindess <ma...@googlemail.com>.
In message <3b...@mail.gmail.com>,
Nathan Beyer writes:
>
> I've been thinking about how we consume Xerces and Xalan, especially
> since we've had to do some of the more recent modifications to the
> build scripts to manipulate the JARs and archives in various ways. As
> an alternative to grabbing the binary packages, we could grab the
> source code itself and do our own builds. We could do this by grabbing
> the officially distributed source archives or we could checkout the
> code directly via svn:externals pointing to release tags (some risk in
> that).
I'd rather not use svn:externals.
> One advantage this has is that the code would be compiled at the
> bytecode level of our code (Xerces currently builds to support Java
> 1.3). The other would be a more natural fit into the classlib module
> structure, which would allow us to build and package manifests as well
> as additional tests.
I've been thinking about creating modules/xml (and modules/orb) so that
these dependencies are handled in a way that is more consistent with
our modular structure. It would allow us to do things like
-Dexclude.module=orb and not have to download the yoko dependency (which
would be sensible for Harmony Select).
> Just something I've been knocking around in my head. Any comments or
> additional thoughts?
We don't really do very much to the jar. (Now that we are doing it only
once per download rather than once per build) I don't think it is a big
deal. However, I like the idea of actually running xerces and xalan
tests on Harmony and perhaps the best way to do that is to grab source -
unless the tests are in a jar?
Regards,
Mark.
Re: JAXB / JAXWS / StAX (Re: xerces and xalan dependencies)
Posted by Jack Cai <gr...@gmail.com>.
CDDL should be acceptable as Tim indicated. Geronimo also use the JAXB RI
under CDDL.
-Jack
2009/5/2 Davanum Srinivas <da...@gmail.com>
> JaxMe was a JAXB1 implementation. There was some effort to start JAXB2 but
> no one jumped it and hence the effort wound down.
>
> There was an effort in Castor as well [1]
>
> -- dims
>
> [1] http://markmail.org/message/dvzkpjhhrqtpkr5u
>
>
> On 05/01/2009 05:12 PM, Jeremias Maerki wrote:
>
>> On 30.04.2009 22:45:18 Davanum Srinivas wrote:
>>
>>> Since we have started talking about stuff inside Java6.
>>>
>>> For StAX, can we use woodstox - http://woodstox.codehaus.org/
>>>
>>
>> Xerces at least started a StAX implementation. It would be worthwhile to
>> check how far along that is.
>>
>> For JAXB, we dont' have any other choice other than JAXB RI -
>>> https://jaxb.dev.java.net/ - Would CDDL be acceptable?
>>>
>>
>> Does anyone know how complete JaxMe is? http://ws.apache.org/jaxme/
>> I'd prefer ASF-grown stuff if feasible.
>>
>> <snip/>
>>
>>
>> Jeremias Maerki
>>
>>
Re: JAXB / JAXWS / StAX (Re: xerces and xalan dependencies)
Posted by Davanum Srinivas <da...@gmail.com>.
JaxMe was a JAXB1 implementation. There was some effort to start JAXB2 but no one jumped it and hence the effort wound down.
There was an effort in Castor as well [1]
-- dims
[1] http://markmail.org/message/dvzkpjhhrqtpkr5u
On 05/01/2009 05:12 PM, Jeremias Maerki wrote:
> On 30.04.2009 22:45:18 Davanum Srinivas wrote:
>> Since we have started talking about stuff inside Java6.
>>
>> For StAX, can we use woodstox - http://woodstox.codehaus.org/
>
> Xerces at least started a StAX implementation. It would be worthwhile to
> check how far along that is.
>
>> For JAXB, we dont' have any other choice other than JAXB RI - https://jaxb.dev.java.net/ - Would CDDL be acceptable?
>
> Does anyone know how complete JaxMe is? http://ws.apache.org/jaxme/
> I'd prefer ASF-grown stuff if feasible.
>
> <snip/>
>
>
> Jeremias Maerki
>
Re: JAXB / JAXWS / StAX (Re: xerces and xalan dependencies)
Posted by Jeremias Maerki <de...@jeremias-maerki.ch>.
On 30.04.2009 22:45:18 Davanum Srinivas wrote:
> Since we have started talking about stuff inside Java6.
>
> For StAX, can we use woodstox - http://woodstox.codehaus.org/
Xerces at least started a StAX implementation. It would be worthwhile to
check how far along that is.
> For JAXB, we dont' have any other choice other than JAXB RI - https://jaxb.dev.java.net/ - Would CDDL be acceptable?
Does anyone know how complete JaxMe is? http://ws.apache.org/jaxme/
I'd prefer ASF-grown stuff if feasible.
<snip/>
Jeremias Maerki
Re: JAXB / JAXWS / StAX (Re: xerces and xalan dependencies)
Posted by Tim Ellison <t....@gmail.com>.
Davanum Srinivas wrote:
> Since we have started talking about stuff inside Java6.
>
> For StAX, can we use woodstox - http://woodstox.codehaus.org/
I see that is dual license, including ALv2 so that's a contender.
> For JAXB, we dont' have any other choice other than JAXB RI -
> https://jaxb.dev.java.net/ - Would CDDL be acceptable?
Only in binary form. According to the (ever draft) 3rd party licensing
policy [1] we can redistribute the binary JAR under CDDL with
appropriate labeling. This is likely the most pragmatic way forward.
[1] http://www.apache.org/legal/3party.html
> Bringing up StAX and JAXB, since JAX-WS needs both.
>
> For JAX-WS, we can do either Apache CXF or Apache Axis2 or let a user
> choose which one to install like they do in Geronimo. Advantage with CXF
> is that it has tooling for code generation as well (similar to wsimport
> and wsgen in the JDK6)
Ack.
Regards,
Tim
> On 04/30/2009 04:25 PM, Jesse Wilson wrote:
>> 2009/4/29 Alexei Fedotov<al...@gmail.com>
>>
>>> Also it needs checking why Android uses different XML parsers (maybe
>>> we can
>>> switch to better ones as well?)
>>>
>>
>> Harmony's XML stack is quite large. We Androids created our own
>> implementation that omits features like XSLT. Our code is built on
>> xmlpull.org, kxml2, and a high-performance native
>> parser<http://android.git.kernel.org/?p=platform/dalvik.git;a=blob;f=libcore/xml/src/main/java/org/apache/harmony/xml/ExpatParser.java;h=60d74b8a5283ad4bd2d66c5866960c391c2340f2;hb=master>
>>
>> .
>>
>> Cheers,
>> Jesse
>>
>> PS: although this code is in the org.apache.harmony package, it isn't
>> Harmony-sourced. We've been using org.apache.harmony as an 'internal
>> implementation' package, which confuses things.
>>
>
JAXB / JAXWS / StAX (Re: xerces and xalan dependencies)
Posted by Davanum Srinivas <da...@gmail.com>.
Since we have started talking about stuff inside Java6.
For StAX, can we use woodstox - http://woodstox.codehaus.org/
For JAXB, we dont' have any other choice other than JAXB RI - https://jaxb.dev.java.net/ - Would CDDL be acceptable?
Bringing up StAX and JAXB, since JAX-WS needs both.
For JAX-WS, we can do either Apache CXF or Apache Axis2 or let a user choose which one to install like they do in
Geronimo. Advantage with CXF is that it has tooling for code generation as well (similar to wsimport and wsgen in the JDK6)
thanks,
dims
On 04/30/2009 04:25 PM, Jesse Wilson wrote:
> 2009/4/29 Alexei Fedotov<al...@gmail.com>
>
>> Also it needs checking why Android uses different XML parsers (maybe we can
>> switch to better ones as well?)
>>
>
> Harmony's XML stack is quite large. We Androids created our own
> implementation that omits features like XSLT. Our code is built on
> xmlpull.org, kxml2, and a high-performance native
> parser<http://android.git.kernel.org/?p=platform/dalvik.git;a=blob;f=libcore/xml/src/main/java/org/apache/harmony/xml/ExpatParser.java;h=60d74b8a5283ad4bd2d66c5866960c391c2340f2;hb=master>
> .
>
> Cheers,
> Jesse
>
> PS: although this code is in the org.apache.harmony package, it isn't
> Harmony-sourced. We've been using org.apache.harmony as an 'internal
> implementation' package, which confuses things.
>
Re: xerces and xalan dependencies
Posted by Tim Ellison <t....@gmail.com>.
Jesse Wilson wrote:
> 2009/4/29 Alexei Fedotov <al...@gmail.com>
>
>> Also it needs checking why Android uses different XML parsers (maybe we can
>> switch to better ones as well?)
>>
>
> Harmony's XML stack is quite large. We Androids created our own
> implementation that omits features like XSLT. Our code is built on
> xmlpull.org, kxml2, and a high-performance native
> parser<http://android.git.kernel.org/?p=platform/dalvik.git;a=blob;f=libcore/xml/src/main/java/org/apache/harmony/xml/ExpatParser.java;h=60d74b8a5283ad4bd2d66c5866960c391c2340f2;hb=master>
> .
>
> Cheers,
> Jesse
>
> PS: although this code is in the org.apache.harmony package, it isn't
> Harmony-sourced. We've been using org.apache.harmony as an 'internal
> implementation' package, which confuses things.
LOL :-)
Re: xerces and xalan dependencies
Posted by Jesse Wilson <je...@google.com>.
2009/4/29 Alexei Fedotov <al...@gmail.com>
> Also it needs checking why Android uses different XML parsers (maybe we can
> switch to better ones as well?)
>
Harmony's XML stack is quite large. We Androids created our own
implementation that omits features like XSLT. Our code is built on
xmlpull.org, kxml2, and a high-performance native
parser<http://android.git.kernel.org/?p=platform/dalvik.git;a=blob;f=libcore/xml/src/main/java/org/apache/harmony/xml/ExpatParser.java;h=60d74b8a5283ad4bd2d66c5866960c391c2340f2;hb=master>
.
Cheers,
Jesse
PS: although this code is in the org.apache.harmony package, it isn't
Harmony-sourced. We've been using org.apache.harmony as an 'internal
implementation' package, which confuses things.
Re: xerces and xalan dependencies
Posted by Alexei Fedotov <al...@gmail.com>.
Nathan, thanks. This is timely and worth discussing.
As for Apache projects, it may worth consuming them along with their
contributors to make Apache product line look more consistent for those who
use Apache libraries. Also it needs checking why Android uses different XML
parsers (maybe we can switch to better ones as well?)
On Thu, Apr 30, 2009 at 4:50 AM, Nathan Beyer <nd...@apache.org> wrote:
> I've been thinking about how we consume Xerces and Xalan, especially
> since we've had to do some of the more recent modifications to the
> build scripts to manipulate the JARs and archives in various ways. As
> an alternative to grabbing the binary packages, we could grab the
> source code itself and do our own builds. We could do this by grabbing
> the officially distributed source archives or we could checkout the
> code directly via svn:externals pointing to release tags (some risk in
> that).
>
> One advantage this has is that the code would be compiled at the
> bytecode level of our code (Xerces currently builds to support Java
> 1.3). The other would be a more natural fit into the classlib module
> structure, which would allow us to build and package manifests as well
> as additional tests.
>
> Just something I've been knocking around in my head. Any comments or
> additional thoughts?
>
--
With best regards / с наилучшими пожеланиями,
Alexei Fedotov / Алексей Федотов,
http://www.telecom-express.ru/
http://people.apache.org/~aaf/
http://harmony.apache.org/
http://code.google.com/p/openmeetings/