Dev doc tree purged

[Kristian Waagan <Kr...@Sun.COM>]

Hello,

I purged the existing doc tree on people.a.o for the dev build today. It 
had some old files in it not being updated anymore due to changes to the 
manuals.
If I have set the permissions correctly, the existing publish script 
should continue to work (after it has been fixed of course, since it 
seems to be down right now). I have also pointed my cron job to the same 
dev directory, and I pushed the Dec 7. build (built by Hudson [1]). It 
should go live in an hour or so.

We should now have two independent processes publishing docs. This takes 
care of the problem when one of them goes down, but not the case where 
one of the processes publish broken docs. The cron job I have written 
has some simple sanity checks (like verifying that all the expected PDFs 
and *-single.html files are there), but it's far from foolproof.

Please ping me if my changes caused trouble, and/or you need my help due 
to wrong file permissions.
The old directory is saved under "dev_original" until everything is 
known to work.


Thanks,
-- 
Kristian

[1] http://hudson.zones.apache.org/hudson/job/Derby-docs/

Re: Dev doc tree purged

[Kristian Waagan <Kr...@Sun.COM>]

Bryan Pendleton wrote:
>> one of the processes publish broken docs. The cron job I have written 
>> has some simple sanity checks (like verifying that all the expected 
>> PDFs and *-single.html files are there), but it's far from foolproof.
>
> Thanks for working on this, Kristian.
>
> With the recent build problems that we had (DERBY-1194, DERBY-4153, 
> DERBY-3518),
> it seemed, at least in my environment, that the build scripts were 
> successfully
> producing the *-single.html files, but the build still was broken due 
> to the
> invalid tags.
>
> I tried looking through the DITA-provided build scripts to see if we
> could specify something on the <pipeline> task that told it to handle
> errors as build-fatal, but didn't immediately find anything.
>
> So it seems that we may need to post-process the build output ourselves
> in order to detect errors such as the ones I introduced in my recent 
> commits.
>
> Is it possible for your tool to search the output for the string
>
>    [pipeline] [Error]
>
> and then treat any occurences of that as a fatal error?

Hi Bryan,

I have tried to implement what you suggested. It's hard to tell if it is 
really working yet, because the errors in the build have been corrected.
In any case, I instructed ant to write the output to file instead of the 
console. I then grep for the string indicating an error and fail the 
build if one or more occurrences are found.

You can see the script(s) at work here:
http://hudson.zones.apache.org/hudson/view/Derby/job/Derby-docs/28/console

Since I instructed ant to log to file instead of stdout, the output 
isn't included in the Hudson console output anymore. I have however 
compressed the file and made it available as an artifact. Btw, 
compressing it reduces the size from a little under 9 MB to around 160 KB :)


PS! We could also have requested yet another plugin to be installed as 
well, but let's see how this works out first.


Regards,
-- 
Kristian

>
> thanks,
>
> bryan
>
>
>

Re: Dev doc tree purged

[Bryan Pendleton <bp...@amberpoint.com>]

> one of the processes publish broken docs. The cron job I have written 
> has some simple sanity checks (like verifying that all the expected PDFs 
> and *-single.html files are there), but it's far from foolproof.

Thanks for working on this, Kristian.

With the recent build problems that we had (DERBY-1194, DERBY-4153, DERBY-3518),
it seemed, at least in my environment, that the build scripts were successfully
producing the *-single.html files, but the build still was broken due to the
invalid tags.

I tried looking through the DITA-provided build scripts to see if we
could specify something on the <pipeline> task that told it to handle
errors as build-fatal, but didn't immediately find anything.

So it seems that we may need to post-process the build output ourselves
in order to detect errors such as the ones I introduced in my recent commits.

Is it possible for your tool to search the output for the string

    [pipeline] [Error]

and then treat any occurences of that as a fatal error?

thanks,

bryan