You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Oliver Deakin <ol...@googlemail.com> on 2009/05/21 12:30:34 UTC
Re: [classlib][build] Need depends.properties to be in ASCII on
non-ASCII platforms
On z/OS, svn (or rather svnkit - the native svn client doesn't work on
z/OS) automatically converts text/plain files into the native encoding
at checkout, which is generally a good thing because we want at least
the native code to be in native encoding. There is only one exception I
have come across - the svn client seems to check the <?xml ... ?> header
tag in xml files for an encoding property. If it is specified UTF-8 (as
in our build files) svn leaves the files in the UTF-8. If the encoding
is not specified, svn will convert the xml file to the native encoding.
I have also tested the rumoured svnkit:charset and svn:charset
properties, but sadly they do not seem to work either. I'm starting to
think that (2) from my original mail may be the best solution - although
it's a little ugly, at least it only gets run at the fetch-depends stage.
Regards,
Oliver
Nathan Beyer wrote:
> Does SVN really do anything to the encoding of a text/plain file? I
> thought the only thing SVN did to files was swap EOL sequences. When
> using SVN across Win, Linux and Mac on a UTF-8 encoded file, I've
> always had the encoding maintained, even though each OS uses a
> different default encoding. I don't suppose this is a z/OS quirk, is
> it?
>
> On Wed, May 20, 2009 at 9:53 AM, Oliver Deakin
> <ol...@googlemail.com> wrote:
>
>> Hi all,
>>
>> I have an issue building on z/OS - a fresh checkout from svn gives us
>> depends.properties in the default platform encoding (EBCDIC). However, when
>> we come to load these properties in depends-common.xml, using the
>> loadproperties Ant task, they are assumed to be in ASCII encoding. The
>> "encoding" property of the Ant task does not actually work correctly (I
>> think this is a bug in Ant which I intend to raise) so it is no help. I was
>> wondering if anybody has any ideas about the best way to tackle this
>> problem? Really I'd like to tag the file in some way so that it is checked
>> out of Subversion in the correct encoding immediately, but there does not
>> seem to be a way to do this. I have come up with a few options:
>>
>> 1) Mark the file binary in svn - not a good solution, means losing revision
>> history, not getting proper diffs etc.
>> 2) Use the Ant copy task with it's outputencoding property set to ISO-8859-1
>> to copy the file to another location, and then use that copy to load the
>> properties from. Delete the file after it has been used (patch would be like
>> [1]). Unfortunately this means that when we run the fetch-depends target we
>> see a lot of copy/delete output (setting verbose="false" for these tasks
>> does not seem to hide the output completely).
>> 3) Same as (2), but don't delete the file afterwards. There would only be
>> one initial copy, then no further copies unless the file is modified.
>> However, this does mean that these is an extra file left in the make
>> directory, which I don't like the idea of.
>>
>> I'd choose to go with (2), but thought I'd see if anyone has any other ideas
>> before I commit the change.
>>
>> Regards,
>> Oliver
>>
>>
>> [1]
>> Index: depends-common.xml
>> ===================================================================
>> --- depends-common.xml (revision 775934)
>> +++ depends-common.xml (working copy)
>> @@ -25,7 +25,8 @@
>> where possible.
>> </description>
>>
>> - <loadproperties srcfile="${basedir}/make/depends.properties">
>> + <copy file="${basedir}/make/depends.properties"
>> tofile="${basedir}/make/depends.properties.ascii"
>> outputencoding="ISO-8859-1" />
>> + <loadproperties srcfile="${basedir}/make/depends.properties.ascii"
>> encoding="ISO-8859-1">
>> <filterchain>
>> <filterreader classname="org.apache.tools.ant.filters.ReplaceTokens">
>> <param type="token" name="${hy.arch}" value=""/>
>> @@ -34,7 +35,8 @@
>> <param type="token" name="${hy.platform.variant}" value=""/>
>> </filterreader>
>> </filterchain>
>> - </loadproperties>
>> + </loadproperties>
>> + <delete file="${basedir}/make/depends.properties.ascii" />
>>
>> <macrodef name="download-one-file">
>> <attribute name="src" />
>>
>> --
>> Oliver Deakin
>> Unless stated otherwise above:
>> IBM United Kingdom Limited - Registered in England and Wales with number
>> 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
>> PO6 3AU
>>
>>
>>
>
>
--
Oliver Deakin
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [classlib][build] Need depends.properties to be in ASCII on
non-ASCII platforms
Posted by Sean Qiu <se...@gmail.com>.
Or we can transform depends.properties to xml format?
Best Regards
Sean, Xiao Xia Qiu
2009/5/21 Oliver Deakin <ol...@googlemail.com>:
> On z/OS, svn (or rather svnkit - the native svn client doesn't work on z/OS)
> automatically converts text/plain files into the native encoding at
> checkout, which is generally a good thing because we want at least the
> native code to be in native encoding. There is only one exception I have
> come across - the svn client seems to check the <?xml ... ?> header tag in
> xml files for an encoding property. If it is specified UTF-8 (as in our
> build files) svn leaves the files in the UTF-8. If the encoding is not
> specified, svn will convert the xml file to the native encoding.
>
> I have also tested the rumoured svnkit:charset and svn:charset properties,
> but sadly they do not seem to work either. I'm starting to think that (2)
> from my original mail may be the best solution - although it's a little
> ugly, at least it only gets run at the fetch-depends stage.
>
> Regards,
> Oliver
>
> Nathan Beyer wrote:
>>
>> Does SVN really do anything to the encoding of a text/plain file? I
>> thought the only thing SVN did to files was swap EOL sequences. When
>> using SVN across Win, Linux and Mac on a UTF-8 encoded file, I've
>> always had the encoding maintained, even though each OS uses a
>> different default encoding. I don't suppose this is a z/OS quirk, is
>> it?
>>
>> On Wed, May 20, 2009 at 9:53 AM, Oliver Deakin
>> <ol...@googlemail.com> wrote:
>>
>>>
>>> Hi all,
>>>
>>> I have an issue building on z/OS - a fresh checkout from svn gives us
>>> depends.properties in the default platform encoding (EBCDIC). However,
>>> when
>>> we come to load these properties in depends-common.xml, using the
>>> loadproperties Ant task, they are assumed to be in ASCII encoding. The
>>> "encoding" property of the Ant task does not actually work correctly (I
>>> think this is a bug in Ant which I intend to raise) so it is no help. I
>>> was
>>> wondering if anybody has any ideas about the best way to tackle this
>>> problem? Really I'd like to tag the file in some way so that it is
>>> checked
>>> out of Subversion in the correct encoding immediately, but there does not
>>> seem to be a way to do this. I have come up with a few options:
>>>
>>> 1) Mark the file binary in svn - not a good solution, means losing
>>> revision
>>> history, not getting proper diffs etc.
>>> 2) Use the Ant copy task with it's outputencoding property set to
>>> ISO-8859-1
>>> to copy the file to another location, and then use that copy to load the
>>> properties from. Delete the file after it has been used (patch would be
>>> like
>>> [1]). Unfortunately this means that when we run the fetch-depends target
>>> we
>>> see a lot of copy/delete output (setting verbose="false" for these tasks
>>> does not seem to hide the output completely).
>>> 3) Same as (2), but don't delete the file afterwards. There would only be
>>> one initial copy, then no further copies unless the file is modified.
>>> However, this does mean that these is an extra file left in the make
>>> directory, which I don't like the idea of.
>>>
>>> I'd choose to go with (2), but thought I'd see if anyone has any other
>>> ideas
>>> before I commit the change.
>>>
>>> Regards,
>>> Oliver
>>>
>>>
>>> [1]
>>> Index: depends-common.xml
>>> ===================================================================
>>> --- depends-common.xml (revision 775934)
>>> +++ depends-common.xml (working copy)
>>> @@ -25,7 +25,8 @@
>>> where possible.
>>> </description>
>>>
>>> - <loadproperties srcfile="${basedir}/make/depends.properties">
>>> + <copy file="${basedir}/make/depends.properties"
>>> tofile="${basedir}/make/depends.properties.ascii"
>>> outputencoding="ISO-8859-1" />
>>> + <loadproperties srcfile="${basedir}/make/depends.properties.ascii"
>>> encoding="ISO-8859-1">
>>> <filterchain>
>>> <filterreader
>>> classname="org.apache.tools.ant.filters.ReplaceTokens">
>>> <param type="token" name="${hy.arch}" value=""/>
>>> @@ -34,7 +35,8 @@
>>> <param type="token" name="${hy.platform.variant}" value=""/>
>>> </filterreader>
>>> </filterchain>
>>> - </loadproperties>
>>> + </loadproperties>
>>> + <delete file="${basedir}/make/depends.properties.ascii" />
>>>
>>> <macrodef name="download-one-file">
>>> <attribute name="src" />
>>>
>>> --
>>> Oliver Deakin
>>> Unless stated otherwise above:
>>> IBM United Kingdom Limited - Registered in England and Wales with number
>>> 741598. Registered office: PO Box 41, North Harbour, Portsmouth,
>>> Hampshire
>>> PO6 3AU
>>>
>>>
>>>
>>
>>
>
> --
> Oliver Deakin
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
> PO6 3AU
>
>