You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Marina Goldburt <ma...@gmail.com> on 2006/07/04 16:45:23 UTC
[classlib]Using APR for Harmony's native link to the OS
Hi,
To support the idea mentioned in the letter
http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200602.mbox/%3C43F06F25.8040304@gmail.com%3E,
I've tried to implement the file I/O part of the Harmony portlib using APR.
The JIRA issue with the patch is
http://issues.apache.org/jira/browse/HARMONY-751.
The main difficulty to implement I/O function using APR was the lack of
modularity in the classlib native code (the other (not portlib) modules
relies on the portlib functions returning OS I/O handles (but APR uses its
own handles)).
Please, review the patch and comment if the efforts to move OS/platform
specific code to APR make sense and can be useful.
Marina
Re: [classlib]Using APR for Harmony's native link to the OS
Posted by Mladen Turk <mt...@apache.org>.
Garrett Rooney wrote:
>
> Anyway, now we're way off topic for this list ;-)
>
Right :)
--
Mladen.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib]Using APR for Harmony's native link to the OS
Posted by Tim Ellison <t....@gmail.com>.
Garrett Rooney wrote:
> On 7/4/06, Mladen Turk <mt...@apache.org> wrote:
>> Garrett Rooney wrote:
>> >
>> > Until a version of APR exists that doesn't require the use of memory
>> > pools, this approach is simply not reasonable to take.
>>
>> I pretty much agree with you.
>> The entire tomcat-native that makes a low level network
>> abstraction is very much slowed down by the fact that we
>> need to create the pool for every object either socket
>> or file.
>>
>> The entire problem would disappear if there would
>> be a simple apr_free.
>
> Unfortunately, it's not really as simple as that. The underlying APR
> allocators are optimized for quick allocation (a comparison and a
> pointer increment in the common case for non-threaded apps, that plus
> a mutex lock (hopefully noncontested) in threaded, IIRC) at the cost
> of not providing any of the infrastructure that would allow free to
> work. Adding that sort of support would be very nontrivial.
>
> Personally, I think a more practical approach is a separate set of
> functions that create APR objects without pools, with pool based
> wrappers built on top of them for use in systems that are built with
> pools in mind.
>
> Don't get me wrong, I love pools in many situations, but there are a
> huge number of applications that can't use APR today because it
> basically requires you to buy into the pool based approach to resource
> management, which just doesn't fit all programs.
>
> Anyway, now we're way off topic for this list ;-)
Right, we did have this discussion a while ago and I thought we had
concluded it was not going to be a great idea.
However, I am interested in discussing how we could use the classlib
portlib on APR to bound the VM/classlib resource usage (e.g. using APR
as a 'virtual OS' sandbox) -- perhaps so it will play nicely as an httpd
module for example. I'll sign-up to the APR dev list...
Regards,
Tim
--
Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib]Using APR for Harmony's native link to the OS
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 7/4/06, Mladen Turk <mt...@apache.org> wrote:
> Garrett Rooney wrote:
> >
> > Until a version of APR exists that doesn't require the use of memory
> > pools, this approach is simply not reasonable to take.
>
> I pretty much agree with you.
> The entire tomcat-native that makes a low level network
> abstraction is very much slowed down by the fact that we
> need to create the pool for every object either socket
> or file.
>
> The entire problem would disappear if there would
> be a simple apr_free.
Unfortunately, it's not really as simple as that. The underlying APR
allocators are optimized for quick allocation (a comparison and a
pointer increment in the common case for non-threaded apps, that plus
a mutex lock (hopefully noncontested) in threaded, IIRC) at the cost
of not providing any of the infrastructure that would allow free to
work. Adding that sort of support would be very nontrivial.
Personally, I think a more practical approach is a separate set of
functions that create APR objects without pools, with pool based
wrappers built on top of them for use in systems that are built with
pools in mind.
Don't get me wrong, I love pools in many situations, but there are a
huge number of applications that can't use APR today because it
basically requires you to buy into the pool based approach to resource
management, which just doesn't fit all programs.
Anyway, now we're way off topic for this list ;-)
-garrett
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib]Using APR for Harmony's native link to the OS
Posted by Mladen Turk <mt...@apache.org>.
Garrett Rooney wrote:
>
> Until a version of APR exists that doesn't require the use of memory
> pools, this approach is simply not reasonable to take.
I pretty much agree with you.
The entire tomcat-native that makes a low level network
abstraction is very much slowed down by the fact that we
need to create the pool for every object either socket
or file.
The entire problem would disappear if there would
be a simple apr_free.
Regards,
Mladen.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib]Using APR for Harmony's native link to the OS
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 7/4/06, Marina Goldburt <ma...@gmail.com> wrote:
> Hi,
>
>
>
> To support the idea mentioned in the letter
> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200602.mbox/%3C43F06F25.8040304@gmail.com%3E,
>
> I've tried to implement the file I/O part of the Harmony portlib using APR.
>
> The JIRA issue with the patch is
> http://issues.apache.org/jira/browse/HARMONY-751.
>
>
>
> The main difficulty to implement I/O function using APR was the lack of
> modularity in the classlib native code (the other (not portlib) modules
> relies on the portlib functions returning OS I/O handles (but APR uses its
> own handles)).
>
>
>
> Please, review the patch and comment if the efforts to move OS/platform
> specific code to APR make sense and can be useful.
For what it's worth, speaking as an APR developer, I really don't
think this makes sense the way APR currently works.
In this patch you're basically taking the approach of allocating a new
pool whenever you need to call into an APR function. This is
inefficient in the normal case (things like where you're doing it to
call stat, then destroying the pool afterwards), since you end up
allocating 8k worth of data to create the pool, only to free it
immediately afterwards. It's worse in other cases, like in
hyfile_open where you create a new pool to hold the file, but then
proceed to leak it, returning from the function without stashing the
pool anywhere, so you can't ever destroy it, it'll stick around until
apr_terminate is called (probably via atexit, although I haven't
looked at the entire patch so I don't know when it's actually being
called).
Until a version of APR exists that doesn't require the use of memory
pools, this approach is simply not reasonable to take. If you want to
work on making it possible to use APR without pools, please feel free
to join us over on dev@apr and discuss the issue, I think it's an
interesting task, and I'd love to talk about it. But until then,
making use of APR in a garbage collected environment like a Java class
library is simply not going to make sense for anything other than
highly specialized situations where the pool overhead is acceptable
(see the Tomcat JNI based APR connector for example).
-garrett
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib]Using APR for Harmony's native link to the OS
Posted by Geir Magnusson Jr <ge...@pobox.com>.
Marina Goldburt wrote:
> The patch is only an experiment to change the internal portlib
> implementation, that, as I think, reveals some design/implementation
> problems of the luni module.
Great - lets discuss that.
>
> I think, it makes sense to move all platform-specific code of the luni into
> the portlib. And in parallel to cooperate with the APR developers "on
> making
> it possible to use APR without pools" - that, of course, the main problem
> that makes the APR not suitable for JVM needs.
>
> By the end of the process We'll have platform independent interface to
> operating system functionality, that covers all the luni needs,
> isolating in
> the portlib and can switch the portlib implementation to APR (if it have
> the
> appropriate memory model by the moment).
Ok - to start, choose one small thing to discuss...
geir
>
> Marina.
>
> On 7/4/06, Mark Hindess <ma...@googlemail.com> wrote:
>>
>>
>> On 4 July 2006 at 13:13, Geir Magnusson Jr <ge...@pobox.com> wrote:
>> >
>> >
>> > Marina Goldburt wrote:
>> > > Hi,
>> > >
>> > >
>> > >
>> > > To support the idea mentioned in the letter
>> > >
>> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200602.mbox/
>>
>> > %3C43F06F25.8040304@gmail.com%3E,
>> > >
>> > >
>> > > I've tried to implement the file I/O part of the Harmony portlib
>> using
>> > > APR.
>> > >
>> > > The JIRA issue with the patch is
>> > > http://issues.apache.org/jira/browse/HARMONY-751.
>> > >
>> > >
>> >
>> > I wouldn't want to tie us to APR like this - I can see us offering an
>> > "APR port" that is a peer to the native Windows and Linux.ia32 ports
>> [it
>> > would be fun to test performance differences], but not make it the
>> core,
>> > shared implementation.
>>
>> +1
>>
>> I'd always imagined it would progress this way until it was in a
>> position to supplant the current ports.
>>
>> -Mark.
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib]Using APR for Harmony's native link to the OS
Posted by Tim Ellison <t....@gmail.com>.
Marina Goldburt wrote:
> The patch is only an experiment to change the internal portlib
> implementation, that, as I think, reveals some design/implementation
> problems of the luni module.
Please elaborate.
I'm pleased that you are looking at this code, but may I suggest that
you make a point of talking to people here early rather than crafting a
large patch in silence. Firstly, it means that we all learn from your
learning experience; and secondly if there is some doubt over the
approach you will hear about it quicker.
Sorry if that sounds negative, but I'd hate to see your hard work and
enthusiasm wasted.
Regards,
Tim
> I think, it makes sense to move all platform-specific code of the luni into
> the portlib. And in parallel to cooperate with the APR developers "on
> making
> it possible to use APR without pools" - that, of course, the main problem
> that makes the APR not suitable for JVM needs.
>
> By the end of the process We'll have platform independent interface to
> operating system functionality, that covers all the luni needs,
> isolating in
> the portlib and can switch the portlib implementation to APR (if it have
> the
> appropriate memory model by the moment).
>
> Marina.
>
> On 7/4/06, Mark Hindess <ma...@googlemail.com> wrote:
>>
>>
>> On 4 July 2006 at 13:13, Geir Magnusson Jr <ge...@pobox.com> wrote:
>> >
>> >
>> > Marina Goldburt wrote:
>> > > Hi,
>> > >
>> > >
>> > >
>> > > To support the idea mentioned in the letter
>> > >
>> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200602.mbox/
>>
>> > %3C43F06F25.8040304@gmail.com%3E,
>> > >
>> > >
>> > > I've tried to implement the file I/O part of the Harmony portlib
>> using
>> > > APR.
>> > >
>> > > The JIRA issue with the patch is
>> > > http://issues.apache.org/jira/browse/HARMONY-751.
>> > >
>> > >
>> >
>> > I wouldn't want to tie us to APR like this - I can see us offering an
>> > "APR port" that is a peer to the native Windows and Linux.ia32 ports
>> [it
>> > would be fun to test performance differences], but not make it the
>> core,
>> > shared implementation.
>>
>> +1
>>
>> I'd always imagined it would progress this way until it was in a
>> position to supplant the current ports.
>>
>> -Mark.
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
--
Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib]Using APR for Harmony's native link to the OS
Posted by Marina Goldburt <ma...@gmail.com>.
The patch is only an experiment to change the internal portlib
implementation, that, as I think, reveals some design/implementation
problems of the luni module.
I think, it makes sense to move all platform-specific code of the luni into
the portlib. And in parallel to cooperate with the APR developers "on making
it possible to use APR without pools" - that, of course, the main problem
that makes the APR not suitable for JVM needs.
By the end of the process We'll have platform independent interface to
operating system functionality, that covers all the luni needs, isolating in
the portlib and can switch the portlib implementation to APR (if it have the
appropriate memory model by the moment).
Marina.
On 7/4/06, Mark Hindess <ma...@googlemail.com> wrote:
>
>
> On 4 July 2006 at 13:13, Geir Magnusson Jr <ge...@pobox.com> wrote:
> >
> >
> > Marina Goldburt wrote:
> > > Hi,
> > >
> > >
> > >
> > > To support the idea mentioned in the letter
> > >
> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200602.mbox/
> > %3C43F06F25.8040304@gmail.com%3E,
> > >
> > >
> > > I've tried to implement the file I/O part of the Harmony portlib
> using
> > > APR.
> > >
> > > The JIRA issue with the patch is
> > > http://issues.apache.org/jira/browse/HARMONY-751.
> > >
> > >
> >
> > I wouldn't want to tie us to APR like this - I can see us offering an
> > "APR port" that is a peer to the native Windows and Linux.ia32 ports [it
> > would be fun to test performance differences], but not make it the core,
> > shared implementation.
>
> +1
>
> I'd always imagined it would progress this way until it was in a
> position to supplant the current ports.
>
> -Mark.
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
Re: [classlib]Using APR for Harmony's native link to the OS
Posted by Geir Magnusson Jr <ge...@pobox.com>.
Marina Goldburt wrote:
> Hi,
>
>
>
> To support the idea mentioned in the letter
> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200602.mbox/%3C43F06F25.8040304@gmail.com%3E,
>
>
> I've tried to implement the file I/O part of the Harmony portlib using
> APR.
>
> The JIRA issue with the patch is
> http://issues.apache.org/jira/browse/HARMONY-751.
>
>
I wouldn't want to tie us to APR like this - I can see us offering an
"APR port" that is a peer to the native Windows and Linux.ia32 ports [it
would be fun to test performance differences], but not make it the core,
shared implementation.
>
> The main difficulty to implement I/O function using APR was the lack of
> modularity in the classlib native code (the other (not portlib) modules
> relies on the portlib functions returning OS I/O handles (but APR uses its
> own handles)).
>
>
> Please, review the patch and comment if the efforts to move OS/platform
> specific code to APR make sense and can be useful.
I'm interested in making APR an option, but I do believe that at least
right now, APR is tailored in part to different usage requirements than
we have in a VM.
What would it take to define the gap, and then working w/ APR to make a
version that fits our needs (as it may other problem domains)?
geir
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org