Ant task using CocoonBean (was Re: New CLI, twice as fast :)

[Upayavira <uv...@upaya.co.uk>]

[Moving from Forrest-dev to cocoon-dev, a discussion between myself, Marc
Portier with input from Nicola Ken]

Marc,

> Nicola just dropped the link in the thread
> http://ant.apache.org/manual/develop.html is basically all you 
> need to get going...

Yup, doesn't look too hard.
 
> > The interface to the bean is pretty simple. What you would want to do is
> > refactor the o.a.c.Main class into an Ant task. Everything you need is
> > either in processXConf() or in main(). The main code you'll need is as
> > simple as:
> > 
> >         CLIListener listener = new Main.CLIListener();
> 
> what does this listener do?

It is much like the Ant listeners, the way that the bean reports back to
the caller, e.g. saying that a page has been successfully generated.

> also something that could be factored out/reused by the ant task?

Possibly. I've got a CLIListener. We could replace it with a
SimpleOutputStreamBeanListener.

> >         CocoonBean cocoon = new CocoonBean();
> >         cocoon.addListener(listener);
> > 
> >         if (line.hasOption(XCONF_OPT)) {
> >             // destDir from command line overrides one in xconf file
> >             destDir = Main.processXConf(cocoon,
> >             line.getOptionValue(XCONF_OPT), destDir);
> >         }

> I assume this processXConf is where the xconf file is consumed? 
> (i.e. interpreted and used to configure the cocoon-bean?)

Yup.
 
> for the ant task it would be nice if we could hand over an 
> inputstream or stringbuffer that holds the xconf text, would an 
> overloaded version of processXConf() be feasible?
> 
> (1st argument the bean, 2nd argument an inputstream on the xconf 
> data)

It wouldn't be hard. But why would you want to do this? Can you explain?

> > 
> >         cocoon.addTargets(line.getArgList(), destDir);
> > 
> 
> line is holding the command-line I presume?

Yes. So line.getArgList() is simply a list of requested Cocoon URIs.

> while this makes perfect sense for cli usage I would not include 
> such a overriding arguments/options notion inside the ant-task

If I understand what you're saying here - I'd reuse the xconf code, but
would ignore the command-line parameters, so actually I wouldn't use this
cocoon.addTargets() method like this.

> in fact since the processXConf is already there to do the 
> interpretation of the xconf I see no reason to duplicate the 
> effort inside ant-configuration-interpretation

Definitely. It is just a question of finding a common way to interpret
XML configuration, as Main.java uses DOM, whilst Ant uses its own methods
(and Cocoon uses Configure objects...)
 
> my point is that each feature you add to the xconf (by changing 
> bean and processXConf) should automatically be available to the 
> ant-task (it should just pass through the xconf, not try to 
> understand any of it IMHO)

I would by far prefer to do it that way if we can find a way.
 
> this will also make sure that people will only need to do one 
> thing: understand how to write cocoon-bean xconf (not a different 
> knowledge to configure the ant-task)

That'd be great. At the moment, the xconf is a CLI xconf. Someone said
that it shouldn't be a part of the bean itself. But we can make it a
helper class, or a factory, that takes in configuration options and
returns a configured bean.

> >         cocoon.initialize();
> >         cocoon.process();
> >         cocoon.dispose();
> > 
> >         listener.outputBrokenLinks();
> 
> ah, starting to understand why this is for...
> this could opt for an additional configuration element on the ant 
> task: location where to dump this

It is already specified in the cli.xconf, so it could be specified in the
ant task. 

> > BeanListener implementation (simply to decide what to do with output).
> > 
> 
> got that now I think, again I hope for re-use: if I could 
> instantiate such listener and call outputBrokenLinks() with the 
> outputstream it needs to send to then your main could call it 
> with System.out or whatever as an argument, and we'ld all be 
> happy :-)

Fair enough.
 
> > So, yes, I'd happily help to get this going - especially if someone
> > provided me/us with a skeleton for the ant task (which I've never done).
> 
> really no big deal,
> check the 'Example' on the link mentioned before and you're over 
> half way: it should be a snap to do something like
> <cocoon in="path-to-xconf" out="path-to-brokenlinks-output" />
> and then abuse the <echo> task as I mentioned previously for the 
> time being (given my current workload you'ld probably get into 
> this before I do, don't hesitate to ask questions though)
> 
> hmpf, I see they are still mentioning the <taskdef> approach.
> Nicola, wasn't there going to be a different way to define custom 
> tasks or am I completely on the wrong track here?
> 
> now, the bigger challenge I see is supporting the inline xml file 
> for the configuration (without getting into CDATA sections like 
> the echo task is doing)
> 
> <cocoon-bean out="..">
>    <xml-config>
>      <cocoon>... xconf inline / abusing ${ant-props} ...</cocoon>
>    </xml-config>
> </cocoon-bean>
> 
> in fact I think a lot of potential ant tasks could benefit from 
> such a nested-xml-config-ant-datatype since more and more 
> existing tools are requiring an XML config and inlining it with 
> the ant build makes sense when you wrap it up in an ant task and 
> want to use the ant-property-substitution
> 
> hm, have to think some more...
> other suggestions meanwhile?

I think that that is the biggie. I can't see now how to share the xconf
code. If we can work that out, I could knock up the ant task pretty
quickly (especially if you're prepared to test it!).
 
> >>> - there is an Ant task in the Cocoon scratchpad
> >>
> >>ah, didn't know... probably not based on the new cocoon-bean 
> >>stuff I presume?

> > No it doesn't. In some ways the code there is probably better than that
> > in the bean, but to my mind it makes sense to create an Ant wrapper to
> > the bean and then develop the bean, than to have multiple instances of
> > crawlers, etc. (I think there are at least three crawlers in the Cocoon
> > CVS. Quite unnecessary).
> 
> same vision here (developing new features on the bean should not 
> even influence the ant task)

Exactly. As is currently the case!

Regards, Upayavira

Re: Ant task using CocoonBean (was Re: New CLI, twice as fast :)

[Marc Portier <mp...@outerthought.org>]

Upayavira wrote:
> On Fri, 25 Jul 2003 14:01:48 +0200, "Marc Portier" <mp...@outerthought.org>
> said:
> 
> <snip/> 
> 
> 
>>well, the current two ways I envisioned for the ant-task to know 
>>about the xconf are as follows:
>>
>>1/
>><cocoonBean in="location-of-xconf" />
>>
>>2/
>><cocoonBean>
>>   <xml-conf>
>>    INLINE XML OF THE XCONF FILE THAT CAN DO
>>    ${ant-props} SUBSTITUTION
>>   </xml-conf>
>></cocoonBean>
> 
> 
> Definitely the latter would be best.
> 

the first is the easier, and both can coexist nicely in your task 
implementation (I think it just makes sense for people having an 
xconf file already, and as explained it could simulate the second 
approach by using the echo/@file trick)


<big-snip about="everything is in the xconf, yeah!" />

I read some on the wiki page now, and start to see more of the 
light even

> 
> So, let me know how to access the contents of a task node as a DOM node,
> and you'll (in time) get your Ant task.
> 
> How's that for a deal?
> 

uh,
I proposed to get into the first approach because this is exactly 
where I need some time to think ;-)

but putting some effort in now: (rough cut thoughts below)

approach 1/ similar hack from the echo task:

anything that is inside your ant-task as free text will be 
reported to you by void addText(String theNestedText)

(possibly called multiple times based on the underlaying SAX 
characters() event I guess... plus, you should make sure the 
ant-properties get replaced)

so easiest is to go for stealing what the echo-task is doing:

     /**
      * Set a multiline message.
      */
     public void addText(String msg) {
         message += getProject().replaceProperties(msg);
     }

check the full code here: 
http://cvs.apache.org/viewcvs.cgi/ant/src/main/org/apache/tools/ant/taskdefs/Echo.java?rev=1.28&content-type=text/vnd.viewcvs-markup

catch: for ant to ignore the nested XML of your 'text' you will 
need to declare the <![CDATA[ ...  ]]> around the nested xconf stuff

after these addText calls your task will have a String containing 
the text (with all properties substituted)

wrap it up in a StringBufferInputStream and hand it over to a 
DOMParser to get your DOMTree

pass that to the processXConf and let your magic do its job...




approach 2/ reusable nested-xml-conf datatype

please let me think some more...



I hope in the mean time the <![CDATA[ hack is an acceptable 
solution... this becomes the ant end-result:

<cocoon><![CDATA[
<cocoon verbose="true"
         follow-links="true"
         precompile-only="false"
         confirm-extensions="false">
    <broken-links type="xml"
                  file="${project.brokenlinkfile}"

....

</cocoon>]]></cocoon>


maybe you now see why I didn't like the ant-task-element to be 
called 'cocoon', but that is really matter of cosmetics
(and of declaring it properly with the <taskdef>)


> Regards, Upayavira
> 

HTH,
-marc=
-- 
Marc Portier                            http://outerthought.org/
Outerthought - Open Source, Java & XML Competence Support Center
Read my weblog at              http://radio.weblogs.com/0116284/
mpo@outerthought.org                              mpo@apache.org

Re: Ant task using CocoonBean (was Re: New CLI, twice as fast :)

[Upayavira <uv...@upaya.co.uk>]

On Fri, 25 Jul 2003 14:01:48 +0200, "Marc Portier" <mp...@outerthought.org>
said:

<snip/> 

> well, the current two ways I envisioned for the ant-task to know 
> about the xconf are as follows:
> 
> 1/
> <cocoonBean in="location-of-xconf" />
> 
> 2/
> <cocoonBean>
>    <xml-conf>
>     INLINE XML OF THE XCONF FILE THAT CAN DO
>     ${ant-props} SUBSTITUTION
>    </xml-conf>
> </cocoonBean>

Definitely the latter would be best.

> in the first case the ant-task would know about the java.io.File
> in the second case the ant-task would probably have a 
> String(Buffer) that holds the nested xconf data
> 
> in both cases the task could present that as a InputStream 
> (FileINputStrea, or StringBufferInputStream) to the processXConf 
> method.
> 
> in fact... typing this reply makes me think that the ant task 
> could very well just have access to the xconf information as an 
> already parsed DOM document in the second case... (as said before 
> this second case is still very vague in my head ATM)

Can you explain how I would get hold of the contents of the <cocoonBean>
element as a DOM document from within the ant task code? That is _all_ I
need to get this working, AFAICS.

(Actually, I'd call the node <cocoon> as the fact that it is a bean is an
implementation fact. The user just wants to run Cocoon.)

> > Yes. So line.getArgList() is simply a list of requested Cocoon URIs.
> > 
> 
> ah, and that is not information that is present in the
> xconf file I persume?

It is present in the xconf, and in a much more sophisticated way. Look at
the cli.xconf file in the root of the Cocoon CVS. It is well commented.
Check specifically the <uris> node.

> >>my point is that each feature you add to the xconf (by changing 
> >>bean and processXConf) should automatically be available to the 
> >>ant-task (it should just pass through the xconf, not try to 
> >>understand any of it IMHO)
> > 
> > I would by far prefer to do it that way if we can find a way.
> 
> I think we're close
> My basic proposition is this: the more we can put into the xconf 
> (without breaking its common sense role) the easier the ant task 
> will adapt to new features (cause it would only need to know 
> about the fact that such file is to be handed over in some way or 
> another)

Yup. Exactly what I have in mind. Just show me the DOM code :-)

> >>this will also make sure that people will only need to do one 
> >>thing: understand how to write cocoon-bean xconf (not a different 
> >>knowledge to configure the ant-task)
> > 
> > That'd be great. At the moment, the xconf is a CLI xconf. Someone said
> > that it shouldn't be a part of the bean itself. But we can make it a
> > helper class, or a factory, that takes in configuration options and
> > returns a configured bean.
> 
> there was some confusion over here as well about this self 
> management, but I consider it a different issue (and I haven't 
> looked enough into it)

I'll make a choice and let people complain.

> >>>        cocoon.initialize();
> >>>        cocoon.process();
> >>>        cocoon.dispose();
> >>>
> >>>        listener.outputBrokenLinks();
> >>
> >>ah, starting to understand why this is for...
> >>this could opt for an additional configuration element on the ant 
> >>task: location where to dump this
> > 
> > It is already specified in the cli.xconf, so it could be specified in the
> > ant task. 
> 
> well, my point was in fact: do not duplicate a thing that already 
> is specified in the xconf.

No, no! It is already in the xconf, and would therefore already be in the
Ant task!
No work required!

> so if the output-location can be specified in the xconf, then I 
> don't see a reason why the ant-task should try to duplicate that 
> effort
> 
> >>>BeanListener implementation (simply to decide what to do with output).
> >>
> >>got that now I think, again I hope for re-use: if I could 
> >>instantiate such listener and call outputBrokenLinks() with the 
> >>outputstream it needs to send to then your main could call it 
> >>with System.out or whatever as an argument, and we'ld all be 
> >>happy :-)
> > 
> > Fair enough.
> >  
> 
> see above: no need for any of this if it can be dumped into the 
> cli.xconf
> 
> (I didn't realize the listener could be aware of specifics buried 
> in the cli.xconf)

Well. At present, the Main.java class reads the xconf and configures the
bean or the listener. The ant task would do the same (using shared helper
classes).

...

> sure enough, the forrest community would get their hands on it 
> pretty fast aswell IMHO

Great.

> (be sure to get them into the loop again when your getting somewhere)

Will do.

So, let me know how to access the contents of a task node as a DOM node,
and you'll (in time) get your Ant task.

How's that for a deal?

Regards, Upayavira

Re: Ant task using CocoonBean (was Re: New CLI, twice as fast :)

[Marc Portier <mp...@outerthought.org>]

Upayavira wrote:
> [Moving from Forrest-dev to cocoon-dev, a discussion between myself, Marc
> Portier with input from Nicola Ken]
> 
> Marc,
> 
> 
>>Nicola just dropped the link in the thread
>>http://ant.apache.org/manual/develop.html is basically all you 
>>need to get going...
> 
> 
> Yup, doesn't look too hard.
>  
> 
>>>The interface to the bean is pretty simple. What you would want to do is
>>>refactor the o.a.c.Main class into an Ant task. Everything you need is
>>>either in processXConf() or in main(). The main code you'll need is as
>>>simple as:
>>>
>>>        CLIListener listener = new Main.CLIListener();
>>
>>what does this listener do?
> 
> 
> It is much like the Ant listeners, the way that the bean reports back to
> the caller, e.g. saying that a page has been successfully generated.
> 
> 
>>also something that could be factored out/reused by the ant task?
> 
> 
> Possibly. I've got a CLIListener. We could replace it with a
> SimpleOutputStreamBeanListener.
> 
> 
>>>        CocoonBean cocoon = new CocoonBean();
>>>        cocoon.addListener(listener);
>>>
>>>        if (line.hasOption(XCONF_OPT)) {
>>>            // destDir from command line overrides one in xconf file
>>>            destDir = Main.processXConf(cocoon,
>>>            line.getOptionValue(XCONF_OPT), destDir);
>>>        }
> 
> 
>>I assume this processXConf is where the xconf file is consumed? 
>>(i.e. interpreted and used to configure the cocoon-bean?)
> 
> 
> Yup.
>  
> 
>>for the ant task it would be nice if we could hand over an 
>>inputstream or stringbuffer that holds the xconf text, would an 
>>overloaded version of processXConf() be feasible?
>>
>>(1st argument the bean, 2nd argument an inputstream on the xconf 
>>data)
> 
> 
> It wouldn't be hard. But why would you want to do this? Can you explain?
> 

well, the current two ways I envisioned for the ant-task to know 
about the xconf are as follows:

1/
<cocoonBean in="location-of-xconf" />

2/
<cocoonBean>
   <xml-conf>
    INLINE XML OF THE XCONF FILE THAT CAN DO
    ${ant-props} SUBSTITUTION
   </xml-conf>
</cocoonBean>


in the first case the ant-task would know about the java.io.File
in the second case the ant-task would probably have a 
String(Buffer) that holds the nested xconf data

in both cases the task could present that as a InputStream 
(FileINputStrea, or StringBufferInputStream) to the processXConf 
method.

in fact... typing this reply makes me think that the ant task 
could very well just have access to the xconf information as an 
already parsed DOM document in the second case... (as said before 
this second case is still very vague in my head ATM)

> 
>>>        cocoon.addTargets(line.getArgList(), destDir);
>>>
>>
>>line is holding the command-line I presume?
> 
> 
> Yes. So line.getArgList() is simply a list of requested Cocoon URIs.
> 

ah, and that is not information that is present in the
xconf file I persume?

> 
>>while this makes perfect sense for cli usage I would not include 
>>such a overriding arguments/options notion inside the ant-task
> 
> 
> If I understand what you're saying here - I'd reuse the xconf code, but
> would ignore the command-line parameters, so actually I wouldn't use this
> cocoon.addTargets() method like this.
> 

nope, I mis-interpreted the getArgList (and ignored the 
meaningfull addTargets name)

if this is not in the cli.xconf file then the ant-task will still 
  need to provide it through these addTargets

(just didn't get that yet)

so this would make the <cocoonBean> task have also some
nested <target> elements, each of them will have the 
addTarget(String) method triggered

however I get the feeling you're saying that this could be in the 
   xconf file as well?
(in which case I would not make special ant-task support)

> 
>>in fact since the processXConf is already there to do the 
>>interpretation of the xconf I see no reason to duplicate the 
>>effort inside ant-configuration-interpretation
> 
> 
> Definitely. It is just a question of finding a common way to interpret
> XML configuration, as Main.java uses DOM, whilst Ant uses its own methods
> (and Cocoon uses Configure objects...)
>  
> 
>>my point is that each feature you add to the xconf (by changing 
>>bean and processXConf) should automatically be available to the 
>>ant-task (it should just pass through the xconf, not try to 
>>understand any of it IMHO)
> 
> 
> I would by far prefer to do it that way if we can find a way.
>  

I think we're close
My basic proposition is this: the more we can put into the xconf 
(without breaking its common sense role) the easier the ant task 
will adapt to new features (cause it would only need to know 
about the fact that such file is to be handed over in some way or 
another)

> 
>>this will also make sure that people will only need to do one 
>>thing: understand how to write cocoon-bean xconf (not a different 
>>knowledge to configure the ant-task)
> 
> 
> That'd be great. At the moment, the xconf is a CLI xconf. Someone said
> that it shouldn't be a part of the bean itself. But we can make it a
> helper class, or a factory, that takes in configuration options and
> returns a configured bean.
> 

there was some confusion over here as well about this self 
management, but I consider it a different issue (and I haven't 
looked enough into it)

> 
>>>        cocoon.initialize();
>>>        cocoon.process();
>>>        cocoon.dispose();
>>>
>>>        listener.outputBrokenLinks();
>>
>>ah, starting to understand why this is for...
>>this could opt for an additional configuration element on the ant 
>>task: location where to dump this
> 
> 
> It is already specified in the cli.xconf, so it could be specified in the
> ant task. 
> 

well, my point was in fact: do not duplicate a thing that already 
is specified in the xconf.


so if the output-location can be specified in the xconf, then I 
don't see a reason why the ant-task should try to duplicate that 
effort

> 
>>>BeanListener implementation (simply to decide what to do with output).
>>>
>>
>>got that now I think, again I hope for re-use: if I could 
>>instantiate such listener and call outputBrokenLinks() with the 
>>outputstream it needs to send to then your main could call it 
>>with System.out or whatever as an argument, and we'ld all be 
>>happy :-)
> 
> 
> Fair enough.
>  

see above: no need for any of this if it can be dumped into the 
cli.xconf

(I didn't realize the listener could be aware of specifics buried 
in the cli.xconf)

> 
>>>So, yes, I'd happily help to get this going - especially if someone
>>>provided me/us with a skeleton for the ant task (which I've never done).
>>
>>really no big deal,
>>check the 'Example' on the link mentioned before and you're over 
>>half way: it should be a snap to do something like
>><cocoon in="path-to-xconf" out="path-to-brokenlinks-output" />
>>and then abuse the <echo> task as I mentioned previously for the 
>>time being (given my current workload you'ld probably get into 
>>this before I do, don't hesitate to ask questions though)
>>
>>hmpf, I see they are still mentioning the <taskdef> approach.
>>Nicola, wasn't there going to be a different way to define custom 
>>tasks or am I completely on the wrong track here?
>>
>>now, the bigger challenge I see is supporting the inline xml file 
>>for the configuration (without getting into CDATA sections like 
>>the echo task is doing)
>>
>><cocoon-bean out="..">
>>   <xml-config>
>>     <cocoon>... xconf inline / abusing ${ant-props} ...</cocoon>
>>   </xml-config>
>></cocoon-bean>
>>
>>in fact I think a lot of potential ant tasks could benefit from 
>>such a nested-xml-config-ant-datatype since more and more 
>>existing tools are requiring an XML config and inlining it with 
>>the ant build makes sense when you wrap it up in an ant task and 
>>want to use the ant-property-substitution
>>
>>hm, have to think some more...
>>other suggestions meanwhile?
> 
> 
> I think that that is the biggie. I can't see now how to share the xconf
> code. If we can work that out, I could knock up the ant task pretty

well only considering the first case (the file-attribute, not the 
inline nesting of xconf) this is roughly what could be done

ant will have the @in attribute map of your task mapped onto a

public void setIn(File xconf) method where you can store the 
reference  (I think File as argument is directly supported)

you can just store your File reference and use it upon the 
execute() of your task to actually open the Inputstream and pass 
  to the (overloaded) processXConf()


if we would have this in place we would be lowering the stress on 
the inline-thing so I (or anyone else) can get some thoughts 
together on that

> quickly (especially if you're prepared to test it!).
>  

sure enough, the forrest community would get their hands on it 
pretty fast aswell IMHO
(be sure to get them into the loop again when your getting somewhere)

> 
>>>>>- there is an Ant task in the Cocoon scratchpad
>>>>
>>>>ah, didn't know... probably not based on the new cocoon-bean 
>>>>stuff I presume?
> 
> 
>>>No it doesn't. In some ways the code there is probably better than that
>>>in the bean, but to my mind it makes sense to create an Ant wrapper to
>>>the bean and then develop the bean, than to have multiple instances of
>>>crawlers, etc. (I think there are at least three crawlers in the Cocoon
>>>CVS. Quite unnecessary).
>>
>>same vision here (developing new features on the bean should not 
>>even influence the ant task)
> 
> 
> Exactly. As is currently the case!
> 
> Regards, Upayavira
> 


regards,
-marc=
-- 
Marc Portier                            http://outerthought.org/
Outerthought - Open Source, Java & XML Competence Support Center
Read my weblog at              http://radio.weblogs.com/0116284/
mpo@outerthought.org                              mpo@apache.org