You are viewing a plain text version of this content. The canonical link for it is here.
Posted to fop-dev@xmlgraphics.apache.org by John Austin <jw...@sympatico.ca> on 2003/11/19 00:39:07 UTC
Re: FOP ~ PropertyList search gives linear performance (FROM:
fop-user)
Looks like I really put my foot into it this time ;-)
I have repeated the measurements I did yesterday and I
think that it is a pretty reasonable conclusion that a
lot of resources are consumed by FOP in its rather
Byzantine property management code.
I just spent a while trying to understand PropertyList
and PropertyListBuilder and found out that I need to
understand Property and Property.Maker as well. I think
I am going to have to help with this part of the project
but it is going to take a while.
I offered (off-line) to look merging the Alt-Design code
in to the main branch but I suspect that there are some
different directions associated with this. Perhaps this
is the reason it has not been done so far. I am still willing
to work on this but I don't want to walk in to a firefight.
I believe the measurements I did yesterday and I feel that
a bit of algorithm replacement should produce a significant
improvement in the program. I would also like to suggest
that anyone interested in performance look at Java Memory
Profiler at http://www.khelekore.org/jmp/performance.html
I suspect there are still major memory leaks in FOP and this
is one tool that will help you track them down.
--
John Austin <jw...@sympatico.ca>
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
Posted by "Peter B. West" <pb...@powerup.com.au>.
No shortages here.
Peter
John Austin wrote:
> On Wed, 2003-11-19 at 19:50, Peter B. West wrote:
>
>>John Austin wrote:
>
> ...
>
>>My apologies to everyone on the list for the testy tone ...
>
> You mean I might have help p*ss*ng people off ?
--
Peter B. West <http://www.powerup.com.au/~pbwest/resume.html>
Re: FOP ~ PropertyList search gives linear
performance (FROM: fop-user)
Posted by John Austin <jw...@sympatico.ca>.
On Wed, 2003-11-19 at 19:50, Peter B. West wrote:
> John Austin wrote:
...
> My apologies to everyone on the list for the testy tone ...
You mean I might have help p*ss*ng people off ?
> Peter
--
John Austin <jw...@sympatico.ca>
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
Posted by "Peter B. West" <pb...@powerup.com.au>.
John Austin wrote:
> On Tue, 2003-11-18 at 22:36, Peter B. West wrote:
>
...
> Overnight I ran another test, using xml-fop_20031118231549.tar.gz
> and the final summary of the call tree SUGGESTS that things are
> better (although the results may be incorrect).
>
...
>
> My focus is on performance. I came to the properties discussion from a
> measurement of program performance. In my experience, measurement is
> the most reliable approach to optimization.
Entirely understandable. Let me know what conclusions you come to. I
will fill you in off-line on the approach I am taking.
My apologies to everyone on the list for the testy tone of my previous
posting.
Peter
--
Peter B. West <http://www.powerup.com.au/~pbwest/resume.html>
Re: FOP ~ PropertyList search gives linear performance
(FROM: fop-user)
Posted by John Austin <jw...@sympatico.ca>.
On Tue, 2003-11-18 at 22:36, Peter B. West wrote:
> That way lies madness. When you look into the *actual* requirements for
> property processing, you may well draw the same conclusions as I did.
> In fact, when I got to the point of running some comparative timing
> tests, I had already decided that there were serious flaws in my design
> which I needed to address before proceeding. Some of those I have
> already mentioned. I will give you the benefit of my experiences
> off-line. You may be able to see a way to integrate the work I have
> done with the current directions of HEAD.
Overnight I ran another test, using xml-fop_20031118231549.tar.gz
and the final summary of the call tree SUGGESTS that things are
better (although the results may be incorrect).
> The last assessment of the situation that I can recall on this list was
> that property handling is working satisfactorily so there is no need to
> pay any attention to it. Yes, really.
My focus is on performance. I came to the properties discussion from a
measurement of program performance. In my experience, measurement is
the most reliable approach to optimization.
I have attached another screen dump. I tyry to keep them small, but
tjhey get a bit grainy if I reduce them too much.
> > I would also like to suggest
> > that anyone interested in performance look at Java Memory
> > Profiler at http://www.khelekore.org/jmp/performance.html
> >
> > I suspect there are still major memory leaks in FOP and this
> > is one tool that will help you track them down.
> >
>
> Peter
--
John Austin <jw...@sympatico.ca>
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
Posted by "Peter B. West" <pb...@powerup.com.au>.
Victor Mote wrote:
> Peter B. West wrote:
>
>
>>The last assessment of the situation that I can recall on this list was
>>that property handling is working satisfactorily so there is no need to
>>pay any attention to it. Yes, really.
>
>
> I don't think this is quite right. It is a matter of priorities. Property
> handling works, layout does not.
OK. Except that I would say, "Property handling works as far as it
goes." If you want a conforming implementation, property handling has
to be complete.
> I can't speak for others, but my hangup on alt-design properties that has
> prevented progress is that it so far has been presented as inextricably
> linked to pull-parsing and the other parts of alt-design. Until either 1)
> the case for that approach has been made more satisfactorily than it has
> until now, or 2) alt-design property handling can be unbundled from
> pull-parsing, it really is too much to ask to take the alt-design approach.
Not if it works. That is a bone of contention, of course. One of the
problems with the Rec, I think, is that only by attempting an
implementation do you gain an understanding of it.
> Especially with the FO Tree isolated, I am glad to have someone step in and
> improve its performance, if (and only if) we nail down its API to the rest
> of FOP so that the rest of FOP doesn't have to know what is going on here.
Peter
--
Peter B. West <http://www.powerup.com.au/~pbwest/resume.html>
RE: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
Posted by Victor Mote <vi...@outfitr.com>.
Peter B. West wrote:
> The last assessment of the situation that I can recall on this list was
> that property handling is working satisfactorily so there is no need to
> pay any attention to it. Yes, really.
I don't think this is quite right. It is a matter of priorities. Property
handling works, layout does not.
I can't speak for others, but my hangup on alt-design properties that has
prevented progress is that it so far has been presented as inextricably
linked to pull-parsing and the other parts of alt-design. Until either 1)
the case for that approach has been made more satisfactorily than it has
until now, or 2) alt-design property handling can be unbundled from
pull-parsing, it really is too much to ask to take the alt-design approach.
Especially with the FO Tree isolated, I am glad to have someone step in and
improve its performance, if (and only if) we nail down its API to the rest
of FOP so that the rest of FOP doesn't have to know what is going on here.
Victor Mote
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
Posted by "Peter B. West" <pb...@powerup.com.au>.
John Austin wrote:
> Looks like I really put my foot into it this time ;-)
What makes you say that?
> I have repeated the measurements I did yesterday and I
> think that it is a pretty reasonable conclusion that a
> lot of resources are consumed by FOP in its rather
> Byzantine property management code.
>
> I just spent a while trying to understand PropertyList
> and PropertyListBuilder and found out that I need to
> understand Property and Property.Maker as well. I think
> I am going to have to help with this part of the project
> but it is going to take a while.
>
> I offered (off-line) to look merging the Alt-Design code
> in to the main branch but I suspect that there are some
> different directions associated with this. Perhaps this
> is the reason it has not been done so far. I am still willing
> to work on this but I don't want to walk in to a firefight.
That way lies madness. When you look into the *actual* requirements for
property processing, you may well draw the same conclusions as I did.
In fact, when I got to the point of running some comparative timing
tests, I had already decided that there were serious flaws in my design
which I needed to address before proceeding. Some of those I have
already mentioned. I will give you the benefit of my experiences
off-line. You may be able to see a way to integrate the work I have
done with the current directions of HEAD.
> I believe the measurements I did yesterday and I feel that
> a bit of algorithm replacement should produce a significant
> improvement in the program.
The last assessment of the situation that I can recall on this list was
that property handling is working satisfactorily so there is no need to
pay any attention to it. Yes, really.
> I would also like to suggest
> that anyone interested in performance look at Java Memory
> Profiler at http://www.khelekore.org/jmp/performance.html
>
> I suspect there are still major memory leaks in FOP and this
> is one tool that will help you track them down.
>
Peter
--
Peter B. West <http://www.powerup.com.au/~pbwest/resume.html>
Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)
Posted by Glen Mazza <gr...@yahoo.com>.
--- John Austin <jw...@sympatico.ca> wrote:
>
> I just spent a while trying to understand
> PropertyList
> and PropertyListBuilder and found out that I need to
> understand Property and Property.Maker as well. I
> think
> I am going to have to help with this part of the
> project
> but it is going to take a while.
>
> I offered (off-line) to look merging the Alt-Design
> code
> in to the main branch but I suspect that there are
> some
> different directions associated with this. Perhaps
> this
> is the reason it has not been done so far. I am
> still willing
> to work on this but I don't want to walk in to a
> firefight.
>
It would be great if you can help us out--you may be
one of the few that can understand both Alt-Design and
our current approach in HEAD--plus you seem to be very
knowledgable in code optimization--what we need here.
May I suggest, start tentatively sketching out your
ideas for properties on our FOP wiki page--you'll
probably get comments from many of the other
committers.
> I believe the measurements I did yesterday and I
> feel that
> a bit of algorithm replacement should produce a
> significant
> improvement in the program. I would also like to
> suggest
> that anyone interested in performance look at Java
> Memory
> Profiler at
> http://www.khelekore.org/jmp/performance.html
>
That looks like a fantastic product for us--I did a
Google search on it as soon as I saw your screen shots
on it--thanks--we possibly should even add it to our
team page so others can critique our code as well.
> I suspect there are still major memory leaks in FOP
> and this
> is one tool that will help you track them down.
>
Yes, this is something that will help us get to
them--Thanks!
Glen
__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree
RE: FOP ~ PropertyList search gives linear performance (FROM:fop-user)
Posted by Victor Mote <vi...@outfitr.com>.
Peter B. West wrote:
> > 1. improved property handling
> > 2. pull-parsing instead of an FO Tree (but with some data
> stored in memory)
> > 3. a different layout approach
...
> > Re #2: this is the one that I don't yet understand. Peter and I
> had a thread
> > about six months ago that tried to resolve this, and I think he
> will respond
> > to this when he has time.
>
> That thread was as much (or more) about #3, IIRC. Yes, alt.design uses
> pull parsing, but it definitely builds a tree, using the Tree and
> related classes that I wrote when I first became involved. The major
> difference is not with pull parsing (although it necessitates a
> different approach for extensions), but with the layout dependencies
> that I see as unavoidable for property resolution. (See below.)
OK. I am using "FO Tree" in a specific sense, i.e. the FOP FO Tree. I think
you are referring to a different data structure of your own making. The
point is that if John or anyone else wants to help integrate your work into
the trunk line of development, either 1) the existing FOP FO Tree has to be
scrapped, or 2) your Tree structure must exist in parallel with the existing
FOP FO Tree. Is that not correct?
> > Re #3: we have now implemented LayoutStrategy, which means that
> alternative
> > layout systems can at least theoretically be dropped into FOP. See
> > subsequent announcement for more information on this aspect.
>
> And thereby hangs a tale. I started (FOP)life as a believer in the
> separation of property resolution and layout as sequential stages, as
> the Rec seems to state very clearly. In the process of trying to code
> the property resolution, it gradually dawned on me that such separation
> cannot occur. Going back to the Rec, I found the editors' escape
> clause. Dialogue with the editors has since confirmed that they were
> aware of the layout dependency for property resolution.
I just spent some time reviewing some of the doc in alt.design again. I
looked for but did not see documentation about your thought process here
(i.e. why FO Tree should be abandoned in favor of your system). I know it is
embedded in the archives, but do you have a well-written summary of the
whole thing somewhere that I could use to refresh my memory? There is a lot
of "what" in the doc -- if you could add some "why" it would be a big help.
> The question for me was, "Do I kludge around this dependency and
> introduce back-door methods of solving the layout feedback issues for
> property resolution, or do I look for a design that is up-front about
> this?" I was engaged in that search. The process will not be simple
> and clean, because the problem is not. I'll settle for coherency,
> robustness and comprehensibility, if I can achieve them.
I will freely admit that I don't know as much about this problem as you do.
Your position is that the devil in the details makes FOP's high-level design
a "kludge". Since I am (regrettably and much to my frustration) knee-deep in
the high-level design issues, I *really, really* want to make sure that we
are on the right track with them. IOW, lets try again to resolve this, if
you have time.
Victor Mote
Re: FOP ~ PropertyList search gives linear performance (FROM:fop-user)
Posted by "Peter B. West" <pb...@powerup.com.au>.
Victor Mote wrote:
> John Austin wrote:
>
>>I offered (off-line) to look merging the Alt-Design code
>>in to the main branch but I suspect that there are some
>>different directions associated with this. Perhaps this
>>is the reason it has not been done so far. I am still willing
>>to work on this but I don't want to walk in to a firefight.
>
>
> FWIW, I don't think there is really a firefight. Our discussions are usually
> at least robust, maybe even rowdy, but AFAICT, there is a large amount of
> mutual respect. That said, we *are* still trying to sort out some design
> issues. We currently have three lines of development, and I have made my
> initial focus an effort to try to get that winnowed down to one as quickly
> as possible. One of the three is Peter's alt-design. Here is a quick
> summary, as I understand it (Peter please correct these if wrong -- I'm not
> trying to misstate your case):
> 1. improved property handling
> 2. pull-parsing instead of an FO Tree (but with some data stored in memory)
> 3. a different layout approach
>
> Re #1: there is general agreement that this can be improved, and performance
> is only one aspect of this. As you have already found, readability is an
> issue as well. Now that we have the FO Tree stuff in trunk isolated (kind
> of), we can begin thinking of the FO Tree as a service or a separate
> product. IOW, it can have its own API "contract" with the rest of FOP. As
> long as the API remains constant, we can change the mechanical aspects
> behind it without breaking anything else. Thus my comments yesterday about
> changing the API first.
>
> Re #2: this is the one that I don't yet understand. Peter and I had a thread
> about six months ago that tried to resolve this, and I think he will respond
> to this when he has time.
That thread was as much (or more) about #3, IIRC. Yes, alt.design uses
pull parsing, but it definitely builds a tree, using the Tree and
related classes that I wrote when I first became involved. The major
difference is not with pull parsing (although it necessitates a
different approach for extensions), but with the layout dependencies
that I see as unavoidable for property resolution. (See below.)
> Re #3: we have now implemented LayoutStrategy, which means that alternative
> layout systems can at least theoretically be dropped into FOP. See
> subsequent announcement for more information on this aspect.
And thereby hangs a tale. I started (FOP)life as a believer in the
separation of property resolution and layout as sequential stages, as
the Rec seems to state very clearly. In the process of trying to code
the property resolution, it gradually dawned on me that such separation
cannot occur. Going back to the Rec, I found the editors' escape
clause. Dialogue with the editors has since confirmed that they were
aware of the layout dependency for property resolution.
The question for me was, "Do I kludge around this dependency and
introduce back-door methods of solving the layout feedback issues for
property resolution, or do I look for a design that is up-front about
this?" I was engaged in that search. The process will not be simple
and clean, because the problem is not. I'll settle for coherency,
robustness and comprehensibility, if I can achieve them.
> The real issue for you John is whether #1 can be separated from #2 within
> the alt-design stuff. I don't know the answer, since properties to me means
> "FO Tree" and means something else in alt-design. I would personally be very
> glad to have you look at the whole thing, being confined neither to the
> status quo nor to Peter's work (but considering both), and make a
> recommendation. One of the reasons for the FO Tree isolation work was to
> make issues like this easier to address -- except for the FO Tree API issues
> (which should be resolved first), you don't have to worry at all about
> breaking layout or renderers.
Peter
--
Peter B. West <http://www.powerup.com.au/~pbwest/resume.html>
Re: FOP ~ PropertyList search gives linear performance (FROM:fop-user)
Posted by "Peter B. West" <pb...@powerup.com.au>.
John Austin wrote:
> On Wed, 2003-11-19 at 15:22, Victor Mote wrote:
>
>>John Austin wrote:
>
>
>>>to work on this but I don't want to walk in to a firefight.
>>
>>FWIW, I don't think there is really a firefight. Our discussions are usually
>>at least robust, maybe even rowdy, but AFAICT, there is a large amount of
>>mutual respect. That said, we *are* still trying to sort out some design
>
>
> Ah yes! The old vigorous and spirited exchange of views.
AKA a full and frank exchange of views.
Peter
--
Peter B. West <http://www.powerup.com.au/~pbwest/resume.html>
RE: FOP ~ PropertyList search gives linear performance
(FROM:fop-user)
Posted by John Austin <jw...@sympatico.ca>.
On Wed, 2003-11-19 at 15:22, Victor Mote wrote:
> John Austin wrote:
> > to work on this but I don't want to walk in to a firefight.
>
> FWIW, I don't think there is really a firefight. Our discussions are usually
> at least robust, maybe even rowdy, but AFAICT, there is a large amount of
> mutual respect. That said, we *are* still trying to sort out some design
Ah yes! The old vigorous and spirited exchange of views.
--
John Austin <jw...@sympatico.ca>
RE: FOP ~ PropertyList search gives linear performance (FROM:fop-user)
Posted by Victor Mote <vi...@outfitr.com>.
John Austin wrote:
> I just spent a while trying to understand PropertyList
> and PropertyListBuilder and found out that I need to
> understand Property and Property.Maker as well. I think
> I am going to have to help with this part of the project
> but it is going to take a while.
Yes, this stuff is a little hard to follow.
> I offered (off-line) to look merging the Alt-Design code
> in to the main branch but I suspect that there are some
> different directions associated with this. Perhaps this
> is the reason it has not been done so far. I am still willing
> to work on this but I don't want to walk in to a firefight.
FWIW, I don't think there is really a firefight. Our discussions are usually
at least robust, maybe even rowdy, but AFAICT, there is a large amount of
mutual respect. That said, we *are* still trying to sort out some design
issues. We currently have three lines of development, and I have made my
initial focus an effort to try to get that winnowed down to one as quickly
as possible. One of the three is Peter's alt-design. Here is a quick
summary, as I understand it (Peter please correct these if wrong -- I'm not
trying to misstate your case):
1. improved property handling
2. pull-parsing instead of an FO Tree (but with some data stored in memory)
3. a different layout approach
Re #1: there is general agreement that this can be improved, and performance
is only one aspect of this. As you have already found, readability is an
issue as well. Now that we have the FO Tree stuff in trunk isolated (kind
of), we can begin thinking of the FO Tree as a service or a separate
product. IOW, it can have its own API "contract" with the rest of FOP. As
long as the API remains constant, we can change the mechanical aspects
behind it without breaking anything else. Thus my comments yesterday about
changing the API first.
Re #2: this is the one that I don't yet understand. Peter and I had a thread
about six months ago that tried to resolve this, and I think he will respond
to this when he has time.
Re #3: we have now implemented LayoutStrategy, which means that alternative
layout systems can at least theoretically be dropped into FOP. See
subsequent announcement for more information on this aspect.
The real issue for you John is whether #1 can be separated from #2 within
the alt-design stuff. I don't know the answer, since properties to me means
"FO Tree" and means something else in alt-design. I would personally be very
glad to have you look at the whole thing, being confined neither to the
status quo nor to Peter's work (but considering both), and make a
recommendation. One of the reasons for the FO Tree isolation work was to
make issues like this easier to address -- except for the FO Tree API issues
(which should be resolved first), you don't have to worry at all about
breaking layout or renderers.
Victor Mote