Re: SVN Inconsitent Line Endings

[David Crossley <cr...@apache.org>]

Ross Gardler wrote:
> Dave Brondsema wrote:
> >
> >I'd say we should find out why the HTML is generated with inconsistent
> >line endings in the first place.
> 
> True.
> 
> I did a little testing, here's what I have discovered so far:
> 
> - any file that is processed by XSLT has the line endings problem
> - on investigating one such file (index.html) I discovered that the 
> generated comments from site2xhtml.xsl (such as breadtrail comment) have 
> CRLF endings whilst most other lines have LF endings.
> 
> I'm running on Windows, and get the same results whether I run under 
> CYGWIN or DOS.
> 
> To reproduce cd into any plugin directory, do "ant docs" and check out 
> the generated docs in build/site.

Does the same behaviour happen if you use Forrest's packaged Ant?
i.e. $FORREST_HOME/tools/ant/bin/ant docs
Also 'unset ANT_HOME' to make it uses Forrest's lib/*.jar

I presume that this does not happen when you do 'forrest' to
build the plugin's docs, rather than doing 'ant docs'.

--David

Re: SVN Inconsitent Line Endings

[David Crossley <cr...@apache.org>]

Ross Gardler wrote:
> David Crossley wrote:
> >Ross Gardler wrote:
> >>David Crossley wrote:
> >>
> >>>I presume that this does not happen when you do 'forrest' to
> >>>build the plugin's docs, rather than doing 'ant docs'.
> >>
> >>I had also assumed that, however it turns our to be incorrect.
> >>
> >>In fact doing "forrest site" on any site, including site-author, results 
> >>in the same error.
> >>
> >>I can only assume that it has not raised it's head before because I have 
> >>not been the first person to commit generated sites to SVN with the eol 
> >>settings set to "native".
> >
> >Sorry, i can't parse that. Too many "nots". :-)
> >
> >All text files should have line-endings appropriate for the local system
> >OS and should always be 'svn propset svn:eol-style native'.
> 
> Sorry, I'll try again, but I'll try by asking a couple of questions
> rather than stating what I think the problem might be.

I am finally beginning to understand what your issues are.
I thought you just had a problem with mixed line-endings,
but it sounds a bigger problem and perhaps separate issues.

> Has anyone built a site using Forrest on Windows and then been the first
> to commit it to SVN?
>
> Do we know for sure that the Ant SVN tasks pick up the SVN config info?

I just reviewed our SVN forrest/site/0.7/docs/plugins/
and found that there are files with *.rss and *.pod
which are missing the 'svn:eol-style native'. So perhaps
your ~/.subversion/config is out-of-date.
http://www.apache.org/dev/version-control.html
http://www.apache.org/dev/svn-eol-style.txt

To test if the Ant task gets the new config,
try 'svn rm' one of the plugins and re-add it.

Those *.rss and *.pod are also not handled by the workaround
below. However, they don't need to be, because there are no
mixed line-endings in those (no xml comments either).

> >>Given the above observation that the XSL generated comments ahve 
> >>different line endings to other XSL generated files I can only assume it 
> >>is something to do with the XSLT parser.
> >
> >What happens when you do a simple test with Xalan on the commandline.
> 
> Not sure, I'll do that soon.
> 
> >>For now I have put a workaround in place in the buildfile (uses an ant 
> >>task to fix the line endings). I'll also place an issue on the issue 
> >>tracker to remind us of this.
> >
> >I saw that. However i couldn't understand why it was converting
> >them to UNIX line-endings regardless of which OS the user is on.
> >Do we need to detect the OS?
> 
> I converted to UNIX more out of habit than design. But it works for me
> and I'm on Windows so I'm happy.

My comment was more an SVN-based one:
"eol-style native" means native to the user's OS.

> >I have not heard of this problem before. At Cocoon and Incubator,
> >people on various operating systems generate the website and commit
> >the result to SVN, with no problems.
> 
> Yes, but they don't do what we are doing with docs generated by Forrest
> and then imported into SVN by ant tasks.

Now i understand the issue a bit more. Sorry, i must have
missed something earlier in the thread.

I don't know if this is related, but thanks Google and mail-archive.com:
"line endings" "xsl:comment" ...
http://issues.apache.org/jira/browse/XALANJ-656
"<xsl:copy> introduces inappropriate line-end characters processing comments".

--David

Re: SVN Inconsitent Line Endings

[Ross Gardler <rg...@apache.org>]

David Crossley wrote:
> Ross Gardler wrote:
> 
>>David Crossley wrote:
>>
>>>I presume that this does not happen when you do 'forrest' to
>>>build the plugin's docs, rather than doing 'ant docs'.
>>
>>I had also assumed that, however it turns our to be incorrect.
>>
>>In fact doing "forrest site" on any site, including site-author, results 
>>in the same error.
>>
>>I can only assume that it has not raised it's head before because I have 
>>not been the first person to commit generated sites to SVN with the eol 
>>settings set to "native".
> 
> 
> Sorry, i can't parse that. Too many "nots". :-)
> 
> All text files should have line-endings appropriate for the local system
> OS and should always be 'svn propset svn:eol-style native'.

Sorry, I'll try again, but I'll try by asking a couple of questions
rather than stating what I think the problem might be.

Has anyone built a site using Forrest on Windows and then been the first
to commit it to SVN?

Do we know for sure that the Ant SVN tasks pick up the SVN config info?

>>Given the above observation that the XSL generated comments ahve 
>>different line endings to other XSL generated files I can only assume it 
>>is something to do with the XSLT parser.
> 
> 
> What happens when you do a simple test with Xalan on the commandline.

Not sure, I'll do that soon.

>>For now I have put a workaround in place in the buildfile (uses an ant 
>>task to fix the line endings). I'll also place an issue on the issue 
>>tracker to remind us of this.
> 
> 
> I saw that. However i couldn't understand why it was converting
> them to UNIX line-endings regardless of which OS the user is on.
> Do we need to detect the OS?

I converted to UNIX more out of habit than design. But it works for me
and I'm on Windows so I'm happy.

> I have not heard of this problem before. At Cocoon and Incubator,
> people on various operating systems generate the website and commit
> the result to SVN, with no problems.

Yes, but they don't do what we are doing with docs generated by Forrest
and then imported into SVN by ant tasks.

Ross

Re: SVN Inconsitent Line Endings

[David Crossley <cr...@apache.org>]

Ross Gardler wrote:
> David Crossley wrote:
> >
> >I presume that this does not happen when you do 'forrest' to
> >build the plugin's docs, rather than doing 'ant docs'.
> 
> I had also assumed that, however it turns our to be incorrect.
> 
> In fact doing "forrest site" on any site, including site-author, results 
> in the same error.
>
> I can only assume that it has not raised it's head before because I have 
> not been the first person to commit generated sites to SVN with the eol 
> settings set to "native".

Sorry, i can't parse that. Too many "nots". :-)

All text files should have line-endings appropriate for the local system
OS and should always be 'svn propset svn:eol-style native'.

> Given the above observation that the XSL generated comments ahve 
> different line endings to other XSL generated files I can only assume it 
> is something to do with the XSLT parser.

What happens when you do a simple test with Xalan on the commandline.

> For now I have put a workaround in place in the buildfile (uses an ant 
> task to fix the line endings). I'll also place an issue on the issue 
> tracker to remind us of this.

I saw that. However i couldn't understand why it was converting
them to UNIX line-endings regardless of which OS the user is on.
Do we need to detect the OS?

I have not heard of this problem before. At Cocoon and Incubator,
people on various operating systems generate the website and commit
the result to SVN, with no problems.

--David

Re: SVN Inconsitent Line Endings

[Ross Gardler <rg...@apache.org>]

David Crossley wrote:
> Ross Gardler wrote:
> 
>>Dave Brondsema wrote:
>>
>>>I'd say we should find out why the HTML is generated with inconsistent
>>>line endings in the first place.
>>
>>True.
>>
>>I did a little testing, here's what I have discovered so far:
>>
>>- any file that is processed by XSLT has the line endings problem
>>- on investigating one such file (index.html) I discovered that the 
>>generated comments from site2xhtml.xsl (such as breadtrail comment) have 
>>CRLF endings whilst most other lines have LF endings.
>>
>>I'm running on Windows, and get the same results whether I run under 
>>CYGWIN or DOS.
>>
>>To reproduce cd into any plugin directory, do "ant docs" and check out 
>>the generated docs in build/site.
> 
> 
> Does the same behaviour happen if you use Forrest's packaged Ant?
> i.e. $FORREST_HOME/tools/ant/bin/ant docs
> Also 'unset ANT_HOME' to make it uses Forrest's lib/*.jar

Yes

> I presume that this does not happen when you do 'forrest' to
> build the plugin's docs, rather than doing 'ant docs'.

I had also assumed that, however it turns our to be incorrect.

In fact doing "forrest site" on any site, including site-author, results 
in the same error.

I can only assume that it has not raised it's head before because I have 
not been the first person to commit generated sites to SVN with the eol 
settings set to "native".

Given the above observation that the XSL generated comments ahve 
different line endings to other XSL generated files I can only assume it 
is something to do with the XSLT parser.

For now I have put a workaround in place in the buildfile (uses an ant 
task to fix the line endings). I'll also place an issue on the issue 
tracker to remind us of this.

Ross