error handling

[Torsten Curdt <tc...@vafer.org>]

I have an interal pipeline which sometimes
throws an exception due to broken links.

   <map:pipeline>
     <map:generate type="html" src="get/{1}"/>
     <map:serialize type="xml"/>
     <map:handle-errors>
       <map:generate src="default.xml"/>
       <map:serialize type="xml"/>
     </map:handle-errors>
   </map:pipeline>

And another one using it

   <map:pipeline>
     <map:generate src="whatever.xml"/>
     <map:transform type="cinclude"/> --> calls the intenal pipeline
     ...
   </map:pipeline>

Using the first pipeline works just
fine. Exceptions are being caught and
a default xml file is being presented
instead.

Unfortunately using it through the
second pipeline (through the cinclude
transformer) brings up the exception.

Anyone a clue what's happing here?

cheers
--
Torsten

RE: error handling

[Antonio Gallardo <ag...@agssa.net>]

Carsten Ziegeler dijo:
> Are you talking about 2.1.5 or CVS Head? CVS Head has a rewritten
> internal request handling that simply might have bugs, so we have
> to fix these bugs first before its really usable.

Yep.!!! it tortures now for 2 days now! :(

Best Regards,

Antonio Gallardo

RE: error handling

[Antonio Gallardo <ag...@agssa.net>]

Carsten Ziegeler dijo:
> Vadim Gritsenko wrote:
>
>>
>> IIRC, last first friday we had a chat with Carsten about
>> adding an attribute to the <handle-errors/> or to the
>> <pipeline/> which will indicate what behavior is needed in
>> case of internal request: throw exception or process
>> <handle-errors/>. Carsten?
>>
> Yes, nothing of that is implemented yet. The discussion started
> because of internal redirects.
> Now, in general the error handlers are never called for internal
> requests - that was a design decision. Starting with 2.1.5 we
> have one exception, internal redirects, so if you do a
> <map:redirect-to uri="cocoon:/something"/> then - and only then -
> the error handler of the internal pipeline is called.

Hi Carsten:

Can you post more about it? Seems like many people, including me ;-), is
having troubles with the cocoon:/ protocol.

Best Regards,

Antonio Gallardo.

Re: error handling

[Sylvain Wallez <sy...@apache.org>]

Torsten Curdt wrote:

>> Sorry to jump in late.
>
>
> Thanks for jumping in at all :)
>
>> There's a technical difficulty, however, as internal requests are 
>> handled differently than external ones when it comes to handling 
>> errors occuring during pipeline execution (not during pipeline 
>> building).
>> - pipelines for external requests are executed as soon as the 
>> pipeline is ended, i.e. in the map:serialize statement, hence under 
>> control of the treeprocessor
>> - pipelines for internal requests are executed when getInputStream() 
>> or toSAX() is called on the "cocoon:" source, out of the control of 
>> the treeprocessor.
>
>
> Ok, but isn't the "cocoon:" source going back to treeprocessor after
> all? Or does it just setup the pipelines? I thought every request is
> goint through the TP.
>
> ...or who is passing the request to the pipeline(s)?


You have to consider the two distinct phases that occur when a request 
is handled:
- building the pipeline (executing sitemap statements) : matchers, 
action and flowscript are called, generator, transformers and serializer 
are added to the pipeline.
- processing the pipeline. The generator's generate() method is called, 
which starts the processing chain.

Building the pipeline is done by the TP. Execution of statements stops 
when encountering a "terminal" statement, i.e. a serialize, read, 
redirect or flowscript call.

Processing the pipeline is different for internal and external requests:
- for external requests, the TP starts the processing within the 
terminal statement (e.g. <serialize>). The enclosing <handle-errors> can 
then be handled correctly.
- for internal requests, the TP does *not* start the processing, but 
gives back a filled pipeline object to the SitemapSource. The 
SitemapSource starts the processing when the content of the source is 
needed. This means that pipeline processing occurs out of the 
<handle-errors> enclosing the <serialize>, and that errors occuring in 
that phase cannot be handled (at least with the current architecture).

>> So we can add add handle-errors="always|external|internal" and 
>> "?cocoon:handle-errors=true", but it will handle errors occuring 
>> during the _building_ of the pipeline, and not during its _execution_.
>
>
> That's not what I am after. It does not help for error handling on 
> aggregation. We should aim for the execution time.


It all depends where the errors occur in the aggregated pipelines: is it 
when building the pipeline or when executing it?

>> Handling errors occuring during the execution of internal requests 
>> would require some not so innocent changes in the pipeline machinery 
>> [3].
>
>
> Well, IMHO this is major flaw and should be tackled.
> No matter if we need to change something or not. I think the pipeline 
> machinery is so deep core that probably not to many people would 
> notice anyway.
>
> Don't know... but maybe it would be possible to move the error 
> handling further down to the pipeline level? If we are able to add 
> that to the Abstract... classes even less people would be affected. 
> But I have no clue if that's possible at all.
>
> Especially if we don't want to mix concerns.


A solution that I proposed in [1] is that for internal processing, the 
TP not only builds the pipeline, but also returns a pointer to the 
error-handling statements wrapped in a Processor. That way, the 
SitemapSource can call the appropriate error handling statements.

>> But we can of course go one step at a time and start by catching 
>> pipeline build-time exceptions.
>
>
> Not sure if that's worth the effort.
>
> I'd propose to change what needs to be changed.


Sure :-)

Sylvain

[1] http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=107876029119774&w=2

-- 
Sylvain Wallez                                  Anyware Technologies
http://www.apache.org/~sylvain           http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }

RE: error handling

[Carsten Ziegeler <cz...@s-und-n.de>]

Torsten Curdt wrote:
> 
> Ok, but isn't the "cocoon:" source going back to 
> treeprocessor after all? Or does it just setup the pipelines? 
> I thought every request is goint through the TP.
> 
The cocoon: protocol is going through the TP for setting up the
pipeline. The assembled pipeline component is returned to the
protocol (Source) and from there directly invoked. So during
execution of the pipeline the TP is not involved at all.

> 
> > So we can add add handle-errors="always|external|internal" and 
> > "?cocoon:handle-errors=true", but it will handle errors occuring 
> > during the _building_ of the pipeline, and not during its 
> _execution_.
> 
> That's not what I am after. It does not help for error 
> handling on aggregation. We should aim for the execution time.
> 
Yepp, execution time is imho more important - if we can do both, great :)

> 
> I'd propose to change what needs to be changed.
> 
Yes :)

I think, Sylvain outlined here

http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=107876029119774&w=2

what has to be done. It should be fairly easy to add this to control
the error handling at execution time. Controlling the error handling
on construction time is very easy as well.

If we add this to 2.2-dev we can change things anyway (even interfaces
if required).

Carsten

Re: error handling

[Torsten Curdt <tc...@vafer.org>]

> Sorry to jump in late.

Thanks for jumping in at all :)

> There's a technical difficulty, however, as internal requests are 
> handled differently than external ones when it comes to handling errors 
> occuring during pipeline execution (not during pipeline building).
> - pipelines for external requests are executed as soon as the pipeline 
> is ended, i.e. in the map:serialize statement, hence under control of 
> the treeprocessor
> - pipelines for internal requests are executed when getInputStream() or 
> toSAX() is called on the "cocoon:" source, out of the control of the 
> treeprocessor.

Ok, but isn't the "cocoon:" source going back to treeprocessor after
all? Or does it just setup the pipelines? I thought every request is
goint through the TP.

...or who is passing the request to the pipeline(s)?

> So we can add add handle-errors="always|external|internal" and 
> "?cocoon:handle-errors=true", but it will handle errors occuring during 
> the _building_ of the pipeline, and not during its _execution_.

That's not what I am after. It does not help for error handling
on aggregation. We should aim for the execution time.

> Handling errors occuring during the execution of internal requests would 
> require some not so innocent changes in the pipeline machinery [3].

Well, IMHO this is major flaw and should be tackled.
No matter if we need to change something or not. I
think the pipeline machinery is so deep core that
probably not to many people would notice anyway.

Don't know... but maybe it would be possible to
move the error handling further down to the pipeline
level? If we are able to add that to the Abstract...
classes even less people would be affected. But
I have no clue if that's possible at all.

Especially if we don't want to mix concerns.

> But we can of course go one step at a time and start by catching 
> pipeline build-time exceptions.

Not sure if that's worth the effort.

I'd propose to change what needs to be changed.

cheers
--
Torsten

Re: error handling

[Sylvain Wallez <sy...@apache.org>]

Torsten Curdt wrote:

>>> ...What about...
>>>
>>>   <map:pipeline handle-errors="always|external|internal"/>
>>>
>>> our current behaviour would be "external".
>>> I would like to have "always" andt the "internal" option might be FS ;)
>>
>>
>> +1 for the declaration, looks clean enough!
>
>
> Carsten, I'd really like to takle this ASAP.
> We are currently using a *very* ugly work around.


Sorry to jump in late.

This problem and possible discussions where discussed at [1], [2] and 
follow-ups. The conclusion was that choosing to handle errors or not in 
internal pipelines could be decided either by the caller or by the 
callee depending on the use cases.

There's a technical difficulty, however, as internal requests are 
handled differently than external ones when it comes to handling errors 
occuring during pipeline execution (not during pipeline building).
- pipelines for external requests are executed as soon as the pipeline 
is ended, i.e. in the map:serialize statement, hence under control of 
the treeprocessor
- pipelines for internal requests are executed when getInputStream() or 
toSAX() is called on the "cocoon:" source, out of the control of the 
treeprocessor.

So we can add add handle-errors="always|external|internal" and 
"?cocoon:handle-errors=true", but it will handle errors occuring during 
the _building_ of the pipeline, and not during its _execution_.

Handling errors occuring during the execution of internal requests would 
require some not so innocent changes in the pipeline machinery [3].

But we can of course go one step at a time and start by catching 
pipeline build-time exceptions.

Sylvain

[1] http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=107874417205088&w=2
[2] http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=107875707604536&w=2
[3] http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=107876029119774&w=2

-- 
Sylvain Wallez                                  Anyware Technologies
http://www.apache.org/~sylvain           http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }

Re: error handling

[Torsten Curdt <tc...@vafer.org>]

>>>>  <map:pipeline handle-errors="always|external|internal"/>
>>>>
>>>>our current behaviour would be "external".
>>>>I would like to have "always" andt the "internal" option 
>>
>>might be FS 
>>
>>>>;)
>>>
>>>
>>>+1 for the declaration, looks clean enough!
>>
>>Carsten, I'd really like to takle this ASAP.
>>We are currently using a *very* ugly work around.
>>
> 
> Yes, yes, do it - you don't have to wait for my +1, I'm just
> a little committer who likes it when everyone ignores him.
> Silence always means: I'm not against it.

alright :)

cheers
--
Torsten

RE: error handling

[Carsten Ziegeler <cz...@s-und-n.de>]

Torsten Curdt wrote: 
> 
> >> ...What about...
> >>
> >>   <map:pipeline handle-errors="always|external|internal"/>
> >>
> >> our current behaviour would be "external".
> >> I would like to have "always" andt the "internal" option 
> might be FS 
> >> ;)
> > 
> > 
> > +1 for the declaration, looks clean enough!
> 
> Carsten, I'd really like to takle this ASAP.
> We are currently using a *very* ugly work around.
> 
Yes, yes, do it - you don't have to wait for my +1, I'm just
a little committer who likes it when everyone ignores him.
Silence always means: I'm not against it.

Carsten

Re: error handling

[Torsten Curdt <tc...@vafer.org>]

>> ...What about...
>>
>>   <map:pipeline handle-errors="always|external|internal"/>
>>
>> our current behaviour would be "external".
>> I would like to have "always" andt the
>> "internal" option might be FS ;)
> 
> 
> +1 for the declaration, looks clean enough!

Carsten, I'd really like to takle this ASAP.
We are currently using a *very* ugly work
around.

WDYT?

cheers
--
Torsten

Re: error handling

[Bertrand Delacretaz <bd...@apache.org>]

Le 4 juin 04, à 09:59, Torsten Curdt a écrit :
> ...What about...
>
>   <map:pipeline handle-errors="always|external|internal"/>
>
> our current behaviour would be "external".
> I would like to have "always" andt the
> "internal" option might be FS ;)

+1 for the declaration, looks clean enough!

-Bertrand

Re: error handling

[Torsten Curdt <tc...@vafer.org>]

> I think we can use the available things.
> 
> We discussed two solutions: specifying it at the call, like:
> 
> cocoon:handle-error:/something or cocoon:/something?cocoon:handle-error=true

Uaaahh ....maybe the first one - but this looks
dead ugly and feels more like hack for what should
be the default.

...or how would you explain a user that a pipeline
behaves differenlty whether it's been called from
the browser or from cocoon?

> (Note the ':' in the second example which is not allowed for
> usual request parameters).

Noted - still ugly ;)

> and specifying it in the called pipeline:
> <map:pipeline handle-errors-for-internal-calls="true"/>

This looks much better

> The names of the attributes/request-parameters I used have not been
> set in the discussions. I just used here some to show the principle.

:-)

What about...

   <map:pipeline handle-errors="always|external|internal"/>

our current behaviour would be "external".
I would like to have "always" andt the
"internal" option might be FS ;)


> Now, implementing this should be a two or three liner :) I already
> had it implemented but lost the code...

You can also give me a hand ...up to you.

cheers
--
Torsten

RE: error handling

[Carsten Ziegeler <cz...@s-und-n.de>]

Torsten Curdt wrote:
> 
> >>IIRC, last first friday we had a chat with Carsten about adding an 
> >>attribute to the <handle-errors/> or to the <pipeline/> which will 
> >>indicate what behavior is needed in case of internal request: throw 
> >>exception or process <handle-errors/>. Carsten?
> 
> Sounds like a good plan. In fact I think it would be a good 
> default behavior but anyway.
> 
> > Yes, nothing of that is implemented yet. The discussion started 
> > because of internal redirects.
> > Now, in general the error handlers are never called for internal 
> > requests - that was a design decision. Starting with 2.1.5 
> we have one 
> > exception, internal redirects, so if you do a <map:redirect-to 
> > uri="cocoon:/something"/> then - and only then - the error 
> handler of 
> > the internal pipeline is called.
> > 
> > HTH?
> 
> Well, it clarifies... ;) Thanks!
> 
> ...but - I think we definitely need a way to handle errors 
> inside internal pipeline calls! Otherwise there is just no 
> way to handle errors in an aggregation appropriately.
> 
> I am happy to do it ...if there are no objections.
> 
> But where to place it? I reckon we either need to make the 
> interal pipeline calls go through the same hook inside the 
> treeprocessor or move the error handling further down into 
> the pipeline code.
> 
I think we can use the available things.

We discussed two solutions: specifying it at the call, like:

cocoon:handle-error:/something or cocoon:/something?cocoon:handle-error=true

(Note the ':' in the second example which is not allowed for
usual request parameters).

and specifying it in the called pipeline:
<map:pipeline handle-errors-for-internal-calls="true"/>

The names of the attributes/request-parameters I used have not been
set in the discussions. I just used here some to show the principle.

Now, implementing this should be a two or three liner :) I already
had it implemented but lost the code...

Carsten

Re: error handling

[Torsten Curdt <tc...@vafer.org>]

>>IIRC, last first friday we had a chat with Carsten about 
>>adding an attribute to the <handle-errors/> or to the 
>><pipeline/> which will indicate what behavior is needed in 
>>case of internal request: throw exception or process 
>><handle-errors/>. Carsten?

Sounds like a good plan. In fact I think it would be
a good default behavior but anyway.

> Yes, nothing of that is implemented yet. The discussion started
> because of internal redirects.
> Now, in general the error handlers are never called for internal
> requests - that was a design decision. Starting with 2.1.5 we
> have one exception, internal redirects, so if you do a 
> <map:redirect-to uri="cocoon:/something"/> then - and only then -
> the error handler of the internal pipeline is called.
> 
> HTH?

Well, it clarifies... ;) Thanks!

...but - I think we definitely need a way to handle errors
inside internal pipeline calls! Otherwise there is just no
way to handle errors in an aggregation appropriately.

I am happy to do it ...if there are no objections.

But where to place it? I reckon we either need to make
the interal pipeline calls go through the same hook
inside the treeprocessor or move the error handling
further down into the pipeline code.

WDYT?

cheers
--
Torsten

RE: error handling

[Carsten Ziegeler <cz...@s-und-n.de>]

Vadim Gritsenko wrote:

> 
> IIRC, last first friday we had a chat with Carsten about 
> adding an attribute to the <handle-errors/> or to the 
> <pipeline/> which will indicate what behavior is needed in 
> case of internal request: throw exception or process 
> <handle-errors/>. Carsten?
> 
Yes, nothing of that is implemented yet. The discussion started
because of internal redirects.
Now, in general the error handlers are never called for internal
requests - that was a design decision. Starting with 2.1.5 we
have one exception, internal redirects, so if you do a 
<map:redirect-to uri="cocoon:/something"/> then - and only then -
the error handler of the internal pipeline is called.

HTH?
Carsten

Re: error handling

[Vadim Gritsenko <va...@reverycodes.com>]

Torsten Curdt wrote:

> I have an interal pipeline which sometimes
> throws an exception due to broken links.
>
>   <map:pipeline>
>     <map:generate type="html" src="get/{1}"/>
>     <map:serialize type="xml"/>
>     <map:handle-errors>
>       <map:generate src="default.xml"/>
>       <map:serialize type="xml"/>
>     </map:handle-errors>
>   </map:pipeline>
>
> And another one using it
>
>   <map:pipeline>
>     <map:generate src="whatever.xml"/>
>     <map:transform type="cinclude"/> --> calls the intenal pipeline
>     ...
>   </map:pipeline>
>
> Using the first pipeline works just
> fine. Exceptions are being caught and
> a default xml file is being presented
> instead.
>
> Unfortunately using it through the
> second pipeline (through the cinclude
> transformer) brings up the exception.
>
> Anyone a clue what's happing here?


IIRC, last first friday we had a chat with Carsten about adding an 
attribute to the <handle-errors/> or to the <pipeline/> which will 
indicate what behavior is needed in case of internal request: throw 
exception or process <handle-errors/>. Carsten?

Vadim

Re: error handling

[Torsten Curdt <tc...@vafer.org>]

After further investigations it seems like exceptions
are passed to the error handler inside the treeprocessor.
But requests using the cocoon protcol don't go through
the treeprocessor the same way as external ones.
So basically error handling is missing - or broken for
such internal requests.

I think this is a vital feature and I am
wondering why noone hit that before.

What do you guys think?

Mr. Treeprocessor (Sylvain) and
Mr. Pipeline (Carsten) in particular ;-)

cheers
--
Torsten

Re: error handling

[Torsten Curdt <tc...@vafer.org>]

> I have an interal pipeline which sometimes
> throws an exception due to broken links.
> 
>   <map:pipeline>
       <map:match pattern="..">
>       <map:generate type="html" src="get/{1}"/>
>       <map:serialize type="xml"/>
       </map:match>
>     <map:handle-errors>
>       <map:generate src="default.xml"/>
>       <map:serialize type="xml"/>
>     </map:handle-errors>
>   </map:pipeline>
> 
> And another one using it
> 
>   <map:pipeline>
       <map:match patern=".."
>       <map:generate src="whatever.xml"/>
>       <map:transform type="cinclude"/> --> calls the intenal pipeline
>     ...
>   </map:pipeline>

...of course

We were discussing it here... does the cocoon
protocol create a new or use the existing
pipeline? ...if it's the same pipeline it
would explain the behaviour.

But IMHO it should work like in the
example above... WDYT?

cheers
--
Torsten