You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by David Van Couvering <Da...@Sun.COM> on 2005/07/19 23:07:31 UTC
Size of common jar file
I'd like to understand the issues and concerns of the size of a common
jar file. Further inspection has shown that pulling in Monitor results
in a large subset of the iapi and impl packages being pulled into the
common area. This will result in a larger footprint for the client. Is
this a showstopper? I'd like to know before I spend too much time
refactoring.
And no, I haven't forgotten about the independent versions of client and
embedded drivers issue :) I'm doing some prototyping which has brought
up this question for me.
Thanks,
David
Re: Size of common jar file
Posted by Jeremy Boynes <jb...@apache.org>.
David Van Couvering wrote:
> Hi, Dan, I've been thinking a lot on what you have to say here. When I
> first looked at this, I noticed what the tools and network server were
> doing for internationalization, and considered duplicating that approach.
>
> But the original motivation was not just internationalization of
> messages, but also the use of SQL States that the exception handling
> framework in the engine does so well. Currently the network driver set
> SQL State to null for all its exceptions. I can also see a lot of use
> for other services in the client, such as the sanity service, the timer
> service to support timeouts, the crypto service to support pluggable
> encryption of data across the wire, some of the utilities in iapi.util,
> etc. And what if we want to have a common logging/tracing framework, or
> if we standardize on some kind of JMX framework? What if we want to add
> J2SE 5 features into the client code -- will we have to build our own
> component model there to support pluggability?
>
> And just in general, it bothers me that we basically have a situation
> where we can't share code, and so the client code has to be written
> independently of the engine, cutting and pasting as need be, as if it
> were a different product altogether.
>
> So although it would be fastidious and I suppose "practical" of me to
> just cut and paste, I really would like to see if we can make this work.
> I feel it would create some leverage that will be very helpful moving
> forward.
>
> But I also don't want to put a lot of work into something that will
> ultimately get voted down by the community. So I'd like to hear what
> others have to say, and I will most happily bow to the will of the group.
>
For all the reasons above, plus others, I think this is a good idea.
--
Jeremy
Re: Size of common jar file
Posted by David Van Couvering <Da...@Sun.COM>.
Hi, Dan. As much as I don't want to let go of this because I think it's
important and we'll need to deal with it, I agree with you that doing
this with new functionality is probably more sane than trying to make it
work with existing functionality.
David
Daniel John Debrunner wrote:
>David Van Couvering wrote:
>
>[lots of good stuff about code sharing]
>
>Maybe a better approach would be to start code sharing with a new
>feature, such as JMX support, and set a pattern using that. Then if
>required the old code could be de-tangled and fitted to the new scheme.
>This might be a cleaner and happier approach, evolution not revolution.
>
>Then we get a good example, working code sharing, a new feature (e.g.
>JMX) and with the least amount of work.
>
>Dan.
>
>
>
>
Re: Using JDK 1.4 features in tests
Posted by Andrew McIntyre <mc...@gmail.com>.
On 7/22/05, Andrew McIntyre <mc...@gmail.com> wrote:
>
> Perl5Util p5u = new Perl5Util();
> Perl5Util.matches("my(reg|ex)", "myinputstring");
hah.
that would be:
p5u.matches("my(reg|ex)", "myinputstring");
X-)
TGIF,
andrew
Re: Using JDK 1.4 features in tests
Posted by David Van Couvering <Da...@Sun.COM>.
OK, great, thanks. I also had the idea to write a JDK 1.4-specific
subclass of the test class and place my format checks in there. Since
the code I know is generic across JDKs I would have been satisfied. But
I think your approach with jakarta-oro sounds better!
David
Andrew McIntyre wrote:
> On 7/22/05, David Van Couvering <Da...@sun.com> wrote:
>
>>Thanks, Andrew. I was trying to add some functionality to look for a
>>pattern in a string, using java.util.regex. In general this test is
>>intended to be run in JDK 1.3, and potentially (I'm not sure) in J2ME.
>>
>>So, am I not allowed to use 1.4 features in these "generic" tests that
>>are intended for use across JVMs? Sounds like it... Sigh...
>
>
> Well, I won't stop you from using 1.4 features in your tests, but I
> suspect you may have a rather difficult time getting them to run
> properly in JDK 1.3 or J2ME. ;-)
>
> Here's the good news: we already require that the jakarta-oro classes
> be available to run the tests. It would be the first instance that I
> know of a test using the oro classes and not just the test harness,
> but they're there and they do pattern matching.
>
> Intead of:
>
> Pattern.matches("my(reg|ex)", charsequence);
>
> import org.apache.oro.text.perl.Perl5Util, and do:
>
> Perl5Util p5u = new Perl5Util();
> Perl5Util.matches("my(reg|ex)", "myinputstring");
>
> Javadoc is available online:
>
> http://jakarta.apache.org/oro/api/
>
> cheers,
> andrew
Re: Using JDK 1.4 features in tests
Posted by Andrew McIntyre <mc...@gmail.com>.
On 7/22/05, David Van Couvering <Da...@sun.com> wrote:
> Thanks, Andrew. I was trying to add some functionality to look for a
> pattern in a string, using java.util.regex. In general this test is
> intended to be run in JDK 1.3, and potentially (I'm not sure) in J2ME.
>
> So, am I not allowed to use 1.4 features in these "generic" tests that
> are intended for use across JVMs? Sounds like it... Sigh...
Well, I won't stop you from using 1.4 features in your tests, but I
suspect you may have a rather difficult time getting them to run
properly in JDK 1.3 or J2ME. ;-)
Here's the good news: we already require that the jakarta-oro classes
be available to run the tests. It would be the first instance that I
know of a test using the oro classes and not just the test harness,
but they're there and they do pattern matching.
Intead of:
Pattern.matches("my(reg|ex)", charsequence);
import org.apache.oro.text.perl.Perl5Util, and do:
Perl5Util p5u = new Perl5Util();
Perl5Util.matches("my(reg|ex)", "myinputstring");
Javadoc is available online:
http://jakarta.apache.org/oro/api/
cheers,
andrew
Re: Using JDK 1.4 features in tests
Posted by David Van Couvering <Da...@Sun.COM>.
Thanks, Andrew. I was trying to add some functionality to look for a
pattern in a string, using java.util.regex. In general this test is
intended to be run in JDK 1.3, and potentially (I'm not sure) in J2ME.
So, am I not allowed to use 1.4 features in these "generic" tests that
are intended for use across JVMs? Sounds like it... Sigh...
David
Andrew McIntyre wrote:
>
> On Jul 22, 2005, at 11:04 AM, David Van Couvering wrote:
>
>> Hi. In some test code, I am getting errors saying the package
>> java.regex is not found. I suspect the tests are being built with JDK
>> 1.3. Is this a requirement -- can the tests not use any JDK 1.4
>> functionality? Or is this just a configuration change I need to make?
>
>
> For each test package, there's a compilation target which compiles the
> classes which use JDK 1.4 functionality. You'll need to make sure that
> your test is included in the target that compiles using
> java14compile.classpath and excluded from the target which compiles
> using compile.classpath. For example, in tests/lang, the target which
> compiles using java14compile.classpath is "compilet2".
>
> But, is this test for functionality that is exposed to JDK 1.3 or
> JSR169? If so, I would think that your test should run in a 1.3 VM or
> J2ME environment.
>
> If the test is for functionality that is only exposed to 1.4 or higher
> VMs, then you should probably add the test to the jdk14 suite to prevent
> it from running in pre-1.4 level environments.
>
> andrew
>
>
Re: Using JDK 1.4 features in tests
Posted by Andrew McIntyre <mc...@gmail.com>.
On Jul 22, 2005, at 11:04 AM, David Van Couvering wrote:
> Hi. In some test code, I am getting errors saying the package
> java.regex is not found. I suspect the tests are being built with
> JDK 1.3. Is this a requirement -- can the tests not use any JDK
> 1.4 functionality? Or is this just a configuration change I need
> to make?
For each test package, there's a compilation target which compiles
the classes which use JDK 1.4 functionality. You'll need to make sure
that your test is included in the target that compiles using
java14compile.classpath and excluded from the target which compiles
using compile.classpath. For example, in tests/lang, the target which
compiles using java14compile.classpath is "compilet2".
But, is this test for functionality that is exposed to JDK 1.3 or
JSR169? If so, I would think that your test should run in a 1.3 VM or
J2ME environment.
If the test is for functionality that is only exposed to 1.4 or
higher VMs, then you should probably add the test to the jdk14 suite
to prevent it from running in pre-1.4 level environments.
andrew
Using JDK 1.4 features in tests
Posted by David Van Couvering <Da...@Sun.COM>.
Hi. In some test code, I am getting errors saying the package
java.regex is not found. I suspect the tests are being built with JDK
1.3. Is this a requirement -- can the tests not use any JDK 1.4
functionality? Or is this just a configuration change I need to make?
Thanks,
David
Re: Size of common jar file
Posted by Daniel John Debrunner <dj...@debrunners.com>.
David Van Couvering wrote:
[lots of good stuff about code sharing]
Maybe a better approach would be to start code sharing with a new
feature, such as JMX support, and set a pattern using that. Then if
required the old code could be de-tangled and fitted to the new scheme.
This might be a cleaner and happier approach, evolution not revolution.
Then we get a good example, working code sharing, a new feature (e.g.
JMX) and with the least amount of work.
Dan.
Re: Size of common jar file
Posted by David Van Couvering <Da...@Sun.COM>.
Hi, Dan, I've been thinking a lot on what you have to say here. When I
first looked at this, I noticed what the tools and network server were
doing for internationalization, and considered duplicating that approach.
But the original motivation was not just internationalization of
messages, but also the use of SQL States that the exception handling
framework in the engine does so well. Currently the network driver set
SQL State to null for all its exceptions. I can also see a lot of use
for other services in the client, such as the sanity service, the timer
service to support timeouts, the crypto service to support pluggable
encryption of data across the wire, some of the utilities in iapi.util,
etc. And what if we want to have a common logging/tracing framework, or
if we standardize on some kind of JMX framework? What if we want to add
J2SE 5 features into the client code -- will we have to build our own
component model there to support pluggability?
And just in general, it bothers me that we basically have a situation
where we can't share code, and so the client code has to be written
independently of the engine, cutting and pasting as need be, as if it
were a different product altogether.
So although it would be fastidious and I suppose "practical" of me to
just cut and paste, I really would like to see if we can make this work.
I feel it would create some leverage that will be very helpful moving
forward.
But I also don't want to put a lot of work into something that will
ultimately get voted down by the community. So I'd like to hear what
others have to say, and I will most happily bow to the will of the group.
Thanks,
David
Daniel John Debrunner wrote:
[snip]
>
>
> David, you may want to step back and see what you are trying to do. I
> think the original goal is to have the messages/exceptions localized
> from the client.
>
> If you look at the other Derby code you see that for message handling
> each time the original code (for the engine) has been cloned so as not
> to share it. Now cloning/copying code is generally bad, but this is
> pretty simple & stable code.
>
> So my viewpoint you seem to be putting a lot of effort in for very
> little gain in value, most likely especially when you see that the exact
> model used by the engine may not be well suited to the client. The
> engine hashes its messages into 50 message files, so as not to have a
> huge Properties object in memory for the messages. Now I would expect
> the client to have few messages so hashing into separate files may not
> be worth it.
>
> So in summary, a shared code plan seems to be a lot of work, introduce
> several problems and solve little.
>
> Dan.
>
Re: Size of common jar file
Posted by Daniel John Debrunner <dj...@debrunners.com>.
>> David Van Couvering wrote:
>>
>>> I'd like to understand the issues and concerns of the size of a
>>> common jar file. Further inspection has shown that pulling in
>>> Monitor results in a large subset of the iapi and impl packages being
>>> pulled into the common area. This will result in a larger footprint
>>> for the client. Is this a showstopper? I'd like to know before I
>>> spend too much time refactoring.
>>>
>>
>> It is concerning but I wouldn't classify as a showstopper. It does
>> perhaps imply more coupling than ideal between the embedded JDBC
>> driver and the underlying engine. Perhaps footprint can be constrained
>> with some small but crucial refactoring?
David, you may want to step back and see what you are trying to do. I
think the original goal is to have the messages/exceptions localized
from the client.
If you look at the other Derby code you see that for message handling
each time the original code (for the engine) has been cloned so as not
to share it. Now cloning/copying code is generally bad, but this is
pretty simple & stable code.
So my viewpoint you seem to be putting a lot of effort in for very
little gain in value, most likely especially when you see that the exact
model used by the engine may not be well suited to the client. The
engine hashes its messages into 50 message files, so as not to have a
huge Properties object in memory for the messages. Now I would expect
the client to have few messages so hashing into separate files may not
be worth it.
So in summary, a shared code plan seems to be a lot of work, introduce
several problems and solve little.
Dan.
Re: JDBC/Engine separation, was: Size of common jar file
Posted by David Van Couvering <Da...@Sun.COM>.
Jeremy Boynes wrote:
> David Van Couvering wrote:
>
>> Hi, Jeremy, I like your idea of a "revolutionary" branch but (a) have
>> no idea how to set one up and (b) I don't think I have the "karma" yet
>> to do it.
>>
>
> a) is easy, just svn copy what you need to a suitable location under
> 'branches' and merge in your local changes. When the revolution is over
> the branch either gets merged to trunk or deleted
>
> To do that you need b) which is blocked waiting for root to create your
> account.
>
OK, thanks
>> I also was planning to do what you call "small but crucial
>> refactoring." First glance was not that promising though. I have
>> been able to keep out all the storage-related services, for the most
>> part.
>>
>> This really isn't about the embedded JDBC driver being tied to the
>> engine. It's more that the services themselves have lived a long life
>> under the assumption that they're running within the engine.
>>
>
> Which services are involved and can we isolate them? I would have
> thought that the JDBC code could use a form of "in-VM" transport to
> separate itself from the engine at a much higher level than the storage
> level.
>
I'll try to summarize all this once I have worked through things a
little more myself.
> I would also wonder if there is a need for an "in-VM, cross-classloader"
> configuration that would allow multiple applications to use different
> JDBC versions but a common engine. I can see use for that when Derby is
> embedded in an IDE like Eclipse or in an appserver where the engine is
> running separate from the applications (e.g. different protection domain).
>
An interesting idea, kind of like local EJBs. My understanding is right
now each application has its own database environment, and you can't
share data across applications in an app server -- going across
classloaders is pretty much identical to trying to go across VMs -- not
allowed if you're running embedded mode.
I may be mistaken, but I don't think you can solve this through futzing
with Java VM semantics. I think you need an "in-memory" version of DRDA
-- perhaps using a memory-mapped NIO buffer as a DRDA message queue...
Way beyond the scope of what I'm working on :)
David
> --
> Jeremy
JDBC/Engine separation, was: Size of common jar file
Posted by Jeremy Boynes <jb...@apache.org>.
David Van Couvering wrote:
> Hi, Jeremy, I like your idea of a "revolutionary" branch but (a) have no
> idea how to set one up and (b) I don't think I have the "karma" yet to
> do it.
>
a) is easy, just svn copy what you need to a suitable location under
'branches' and merge in your local changes. When the revolution is over
the branch either gets merged to trunk or deleted
To do that you need b) which is blocked waiting for root to create your
account.
> I also was planning to do what you call "small but crucial refactoring."
> First glance was not that promising though. I have been able to keep
> out all the storage-related services, for the most part.
>
> This really isn't about the embedded JDBC driver being tied to the
> engine. It's more that the services themselves have lived a long life
> under the assumption that they're running within the engine.
>
Which services are involved and can we isolate them? I would have
thought that the JDBC code could use a form of "in-VM" transport to
separate itself from the engine at a much higher level than the storage
level.
I would also wonder if there is a need for an "in-VM, cross-classloader"
configuration that would allow multiple applications to use different
JDBC versions but a common engine. I can see use for that when Derby is
embedded in an IDE like Eclipse or in an appserver where the engine is
running separate from the applications (e.g. different protection domain).
--
Jeremy
Re: Size of common jar file
Posted by David Van Couvering <Da...@Sun.COM>.
Hi, Jeremy, I like your idea of a "revolutionary" branch but (a) have no
idea how to set one up and (b) I don't think I have the "karma" yet to
do it.
I also was planning to do what you call "small but crucial refactoring."
First glance was not that promising though. I have been able to keep
out all the storage-related services, for the most part.
This really isn't about the embedded JDBC driver being tied to the
engine. It's more that the services themselves have lived a long life
under the assumption that they're running within the engine.
David
Jeremy Boynes wrote:
> David Van Couvering wrote:
>
>> I'd like to understand the issues and concerns of the size of a common
>> jar file. Further inspection has shown that pulling in Monitor
>> results in a large subset of the iapi and impl packages being pulled
>> into the common area. This will result in a larger footprint for the
>> client. Is this a showstopper? I'd like to know before I spend too
>> much time refactoring.
>>
>
> It is concerning but I wouldn't classify as a showstopper. It does
> perhaps imply more coupling than ideal between the embedded JDBC driver
> and the underlying engine. Perhaps footprint can be constrained with
> some small but crucial refactoring?
>
> I would suggest checking what you have into a "revolutionary" branch so
> that more people can be involved and help drive a solution.
>
> --
> Jeremy
Re: Size of common jar file
Posted by Jeremy Boynes <jb...@apache.org>.
David Van Couvering wrote:
> I'd like to understand the issues and concerns of the size of a common
> jar file. Further inspection has shown that pulling in Monitor results
> in a large subset of the iapi and impl packages being pulled into the
> common area. This will result in a larger footprint for the client. Is
> this a showstopper? I'd like to know before I spend too much time
> refactoring.
>
It is concerning but I wouldn't classify as a showstopper. It does
perhaps imply more coupling than ideal between the embedded JDBC driver
and the underlying engine. Perhaps footprint can be constrained with
some small but crucial refactoring?
I would suggest checking what you have into a "revolutionary" branch so
that more people can be involved and help drive a solution.
--
Jeremy