How much work is left before we can release HEAD?

[Chris Bowditch <bo...@hotmail.com>]

Team,

I have been trying to work out what is left to do be done before we can do an 
initial release of HEAD, 0.3, say. I know some of you will prefer to aim for a 
1.0 and get everything right first time, but please bear with me.

I have consolidated the layout issues from [1] and [2] The infrastructure 
items listed in [1] I feel are good enough for a 0.3 release. This is largely 
thanks to work from Glen, Finn, Simon, Luca, Peter, Jeremias and the other 
committers. Sorry if Ive missed anyone.

Anyway, i have created a wiki containing the work items along with my opinion 
of priority and a finger in the air time estimates:

http://nagoya.apache.org/wiki/apachewiki.cgi?FOPWorkEstimates

I would very much appreciate some feedback.

Chris

[1] http://nagoya.apache.org/wiki/apachewiki.cgi?FOPProjectTasks

[2] http://xml.apache.org/fop/design/layout.html#status-todo

Re: How much work is left before we can release HEAD?

[Glen Mazza <gr...@yahoo.com>]

> I have also begun using a small extension element to
> make junit like 
> testing of the property subsystem. I've attached the
> source just to show 
> how I intent to do it.

<fox:assert/> looks like a good idea.

I wonder:  would this eventually be better as an
"extension property"--using XPATH, perhaps?  (This
would certainly be well beyond my coding capabilities,
however.)

Glen

Re: How much work is left before we can release HEAD?

[Finn Bock <bc...@worldonline.dk>]

[Jeremias Maerki]

> Those who currently work on layout, how do you choose your work area?

I pick issues based on differences in output from the NIST testsuite.

> One big problem I currently see is testing properly. We don't have a
> good set of tests that we can simply run. The example document are all a
> big mess demonstrating several features at once. Sometimes I don't even
> understand how it should (!) look. Personally, I'd add one important,
> high priority task to the list: (Finally) creating a good test/QS
> environment along with several simple documents each training a single
> feature. Attached to this task should/could be the Java2D renderer which
> we can used to easily create comparable bitmaps. I don't believe in MD5
> checking of PDF at this stage. That may be good as soon as we're in the
> maintenance phase again.

My PDF compare environment is a huge hack, but I have gotten fantastic 
mileage out of the NIST test suite.

I use TestConverter to create a .pdf for each test, use GhostScript to 
create .png for each test, compare .png's and use iText to create a huge 
'diff' .pdf file with the .pdf pages for the tests where the .png differs.

I have also begun using a small extension element to make junit like 
testing of the property subsystem. I've attached the source just to show 
how I intent to do it.

regards,
finn

Re: How much work is left before we can release HEAD?

[Clay Leeds <cl...@medata.com>]

On Sep 28, 2004, at 2:08 AM, Chris Bowditch wrote:
> Simon Pepping wrote:
>> I do not know if this is a lot of work or not. To me it seems a
>> lot. Perhaps you may argue that this is not required for a 0.3
>> release. But to me it is rather essential to the new design. Without
>> it we might as well remain with the maintenance layout system.
>
> keeps and breaks are essential for a 0.3 release. As you said yourself 
> they are the whole reason the redesign was started in the first place. 
> I'm not sure block stacking constraints are essential.

IMO, 0.20.5 functionality should be met in a 0.3 release. In my 
estimation, keeps and breaks are not essential for a 0.3 release as 
long as keeps work for table-rows (i.e., mimics 0.20.5 functionality), 
and breaks work the same as 0.20.5 as well. 0.3 should not be a 'step 
backward'.

Web Maestro Clay

Re: How much work is left before we can release HEAD?

[Chris Bowditch <bo...@hotmail.com>]

Simon Pepping wrote:

<snip/>

> My interest in FOP's layout is mostly theoretical. I cannot get
> enthousiastic about todo lists, time schedules and time estimates.

Thats understandable.

> 
> I would like to see keep and break properties implemented. They are
> the raison d'être of the new design. I do not think they can be
> implemented with the current design, because there is no arbitrator of
> the page length. The problem should be solved in a manner similar to
> the way Luca solved the inline layout problem: All possible break
> points should be returned to a high-level object, probably the Flow
> LM. This then applies a certain algorithm, keeping lengths and keeps
> and breaks into account, to determine the best break point. The
> structure for this procedure must be added to the current design.

Oh I thought this was one of the key points that Keiron addressed when he 
wrote the layout code in HEAD. Sounds like my estimate may be a bit longer then.

> 
> I am also interested in block stacking constraints. They exercise the
> ability to relate the layout produced by one LM with the traits of the
> areas produced by other LMs. Perhaps it can be done using the layout
> context, perhaps one should navigate the LM tree to gather the
> required data.
> 
> Finally I hope it will be possible to make the structure of the layout
> module simpler. I believe it is the complexity of this module that has
> hindered progress the most. With my recent change to LMIter I tried to
> come closer to the semantics of a collection and an iterator, and make
> it easier to create more iterator objects for the children of an LM
> without disturbing the state of the one used in getNextBreakPoss. I
> hope more such changes are possible and will make it easier to
> understand the layout module.

Simplifying the LMIter objects is one way Joerg identified of making layout 
easier to work on.

> 
> I do not know if this is a lot of work or not. To me it seems a
> lot. Perhaps you may argue that this is not required for a 0.3
> release. But to me it is rather essential to the new design. Without
> it we might as well remain with the maintenance layout system.

keeps and breaks are essential for a 0.3 release. As you said yourself they 
are the whole reason the redesign was started in the first place. I'm not sure 
block stacking constraints are essential.

> 
> I understand that the new design for renderers has been more
> successful, and may be a reason to want a release of the development
> code.

This is not a good enough reason for a release.

<snip/>

Chris

Re: How much work is left before we can release HEAD?

[Simon Pepping <sp...@leverkruid.nl>]

On Fri, Sep 24, 2004 at 05:00:45PM +0200, Jeremias Maerki wrote:
> Chris,
> 
> Those who currently work on layout, how do you choose your work area?

My interest in FOP's layout is mostly theoretical. I cannot get
enthousiastic about todo lists, time schedules and time estimates.

I would like to see keep and break properties implemented. They are
the raison d'être of the new design. I do not think they can be
implemented with the current design, because there is no arbitrator of
the page length. The problem should be solved in a manner similar to
the way Luca solved the inline layout problem: All possible break
points should be returned to a high-level object, probably the Flow
LM. This then applies a certain algorithm, keeping lengths and keeps
and breaks into account, to determine the best break point. The
structure for this procedure must be added to the current design.

I am also interested in block stacking constraints. They exercise the
ability to relate the layout produced by one LM with the traits of the
areas produced by other LMs. Perhaps it can be done using the layout
context, perhaps one should navigate the LM tree to gather the
required data.

Finally I hope it will be possible to make the structure of the layout
module simpler. I believe it is the complexity of this module that has
hindered progress the most. With my recent change to LMIter I tried to
come closer to the semantics of a collection and an iterator, and make
it easier to create more iterator objects for the children of an LM
without disturbing the state of the one used in getNextBreakPoss. I
hope more such changes are possible and will make it easier to
understand the layout module.

I do not know if this is a lot of work or not. To me it seems a
lot. Perhaps you may argue that this is not required for a 0.3
release. But to me it is rather essential to the new design. Without
it we might as well remain with the maintenance layout system.

I understand that the new design for renderers has been more
successful, and may be a reason to want a release of the development
code.
 
> One big problem I currently see is testing properly. We don't have a
> good set of tests that we can simply run. The example document are all a
> big mess demonstrating several features at once. Sometimes I don't even
> understand how it should (!) look. Personally, I'd add one important,
> high priority task to the list: (Finally) creating a good test/QS
> environment along with several simple documents each training a single
> feature. Attached to this task should/could be the Java2D renderer which
> we can used to easily create comparable bitmaps. I don't believe in MD5
> checking of PDF at this stage. That may be good as soon as we're in the
> maintenance phase again.

I agree. When I make a change to the layout system that might have
far-ranging bad effects if I miss something, I try to convince myself
with extensive logging. Good tests would be more
satisfactory. Unfortunately, I do not yet know anything about unit
testing, so I cannot write such tests.

Regards, Simon

-- 
Simon Pepping
home page: http://www.leverkruid.nl

Re: How much work is left before we can release HEAD?

[Chris Bowditch <bo...@hotmail.com>]

Jeremias Maerki wrote:

> Chris,
> 
> thank you doing this. I think it's important to have a good instrument
> to determine our progress towards an initial release. I was a bit
> shocked when I summed up only the points you've marked as high priority:
> 20 weeks. It got me thinking and running all the example files again
> that I haven't run since early this year. Some still don't work, some
> don't work anymore. Some look surprisingly good.

Thats what Ive been saying for a while now. I really dont think theres much 
left to do before were ready for a 0.3. Of course those estimates are just 
finger in the air job, so feel free to update the wiki if you spot an estimate 
thats way off.

> 
> To your task list: Do you propose that all listed tasks should be
> completed prior to an initial release? I think that there is a number of
> tasks that are not really necessary for an 0.3, ex. floats, writing
> mode/BIDI, last page. It would be great if we could break down the
> things that need to be done into milestones. That may make it easier to
> communicate to the outside world what our progress is. And we would also
> know where we stand, because that's one of the biggest problem we
> currently have.

Actually I was thinking along the lines of:

High Priorities for 0.3
Mediums for 0.4
Lows for 0.5

The Wiki could really do with re-formatting into a table. I tried using "||" 
but couldnt get the proper effect.

> 
> Those who currently work on layout, how do you choose your work area?
> 
> One big problem I currently see is testing properly. We don't have a
> good set of tests that we can simply run. The example document are all a
> big mess demonstrating several features at once. Sometimes I don't even
> understand how it should (!) look. 

Yes, I have that trouble sometimes as well.

> Personally, I'd add one important,
> high priority task to the list: (Finally) creating a good test/QS
> environment along with several simple documents each training a single
> feature. Attached to this task should/could be the Java2D renderer which
> we can used to easily create comparable bitmaps. I don't believe in MD5
> checking of PDF at this stage. That may be good as soon as we're in the
> maintenance phase again.

Yes, I agree we need a specific testing task. So ill add that to the Wiki. One 
thing we can do when were nearly ready for a release, I can install the 
release one of mt company servers, which see a number of complex documents 
being fired at it.

> 
> Every now and then we get asked when there will be a next release. We
> need to have some kind of answer for them. A good answer may even make
> some company boss invest into FOP because he sees the end of the tunnel.
> I think we will never get there if we target the full feature set for
> the initial release. We can't but break down the whole thing into
> manageable parts.

Yes, we can point them at the FOP Work Estimates Wiki for now.

> 
> Food for flames....as usual.

<snip/>

Chris

Re: How much work is left before we can release HEAD?

[Jeremias Maerki <de...@greenmail.ch>]

Chris,

thank you doing this. I think it's important to have a good instrument
to determine our progress towards an initial release. I was a bit
shocked when I summed up only the points you've marked as high priority:
20 weeks. It got me thinking and running all the example files again
that I haven't run since early this year. Some still don't work, some
don't work anymore. Some look surprisingly good.

To your task list: Do you propose that all listed tasks should be
completed prior to an initial release? I think that there is a number of
tasks that are not really necessary for an 0.3, ex. floats, writing
mode/BIDI, last page. It would be great if we could break down the
things that need to be done into milestones. That may make it easier to
communicate to the outside world what our progress is. And we would also
know where we stand, because that's one of the biggest problem we
currently have.

Those who currently work on layout, how do you choose your work area?

One big problem I currently see is testing properly. We don't have a
good set of tests that we can simply run. The example document are all a
big mess demonstrating several features at once. Sometimes I don't even
understand how it should (!) look. Personally, I'd add one important,
high priority task to the list: (Finally) creating a good test/QS
environment along with several simple documents each training a single
feature. Attached to this task should/could be the Java2D renderer which
we can used to easily create comparable bitmaps. I don't believe in MD5
checking of PDF at this stage. That may be good as soon as we're in the
maintenance phase again.

Every now and then we get asked when there will be a next release. We
need to have some kind of answer for them. A good answer may even make
some company boss invest into FOP because he sees the end of the tunnel.
I think we will never get there if we target the full feature set for
the initial release. We can't but break down the whole thing into
manageable parts.

Food for flames....as usual.

On 22.09.2004 16:35:51 Chris Bowditch wrote:
> Team,
> 
> I have been trying to work out what is left to do be done before we can do an 
> initial release of HEAD, 0.3, say. I know some of you will prefer to aim for a 
> 1.0 and get everything right first time, but please bear with me.
> 
> I have consolidated the layout issues from [1] and [2] The infrastructure 
> items listed in [1] I feel are good enough for a 0.3 release. This is largely 
> thanks to work from Glen, Finn, Simon, Luca, Peter, Jeremias and the other 
> committers. Sorry if Ive missed anyone.
> 
> Anyway, i have created a wiki containing the work items along with my opinion 
> of priority and a finger in the air time estimates:
> 
> http://nagoya.apache.org/wiki/apachewiki.cgi?FOPWorkEstimates
> 
> I would very much appreciate some feedback.
> 
> Chris
> 
> [1] http://nagoya.apache.org/wiki/apachewiki.cgi?FOPProjectTasks
> 
> [2] http://xml.apache.org/fop/design/layout.html#status-todo



Jeremias Maerki