You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Steven Parkes <st...@esseff.org> on 2007/07/17 19:43:52 UTC
binary at the front of CHANGES.txt
Can we get rid of the binary characters at the front of CHANGES.txt? Or
do the mean something?
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: binary at the front of CHANGES.txt
Posted by Yonik Seeley <yo...@apache.org>.
On 7/18/07, DM Smith <dm...@gmail.com> wrote:
> But, the junk at the beginning of the file was C2 BF. Not at all sure
> what this would be.
As I said in my first reply, it *was* a UTF-8 BOM (look back at older
revisions), but I think one of my edits mangled it (I don't recall
what editor I used).
-Yonik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: binary at the front of CHANGES.txt
Posted by DM Smith <dm...@gmail.com>.
On Jul 17, 2007, at 8:40 PM, Yonik Seeley wrote:
> On 7/17/07, DM Smith <dm...@gmail.com> wrote:
>> According to the UTF-8 spec \uFEFF is not a BOM. In UTF-8 the byte
>> order is always the same.
>
> But there is a BOM for UTF-8 (even though there is no endian
> component, it does serve as a marker indicating the text file is
> unicode text encoded in UTF-8).
>
> http://unicode.org/faq/utf_bom.html#29
This is all rather academic at this point as you have fixed the problem.
I stand corrected \uFEFF (the code point) is the BOM for all UTF,
with its representation differing by encoding. But UTF-8 byte order
is always the same, regardless of the presence of the BOM.
According to the Unicode 5.0 Standard book, Chapter 13, Section 13.6,
the byte sequence of the BOM for UTF-8 is EF BB BF (3 bytes) and for
UTF-16 it is FE FF or FF FE (2 bytes). It appears that the byte
sequence is unique for each unicode representation.
See http://www.unicode.org/unicode/uni2book/ch13.pdf#BOM
I frequently will see FE FF at the beginning of UTF-8 files. I have
only seen MS editors add this. This is wrong for UTF-8 files. I was
assuming that this was the junk at the beginning of the file.
But, the junk at the beginning of the file was C2 BF. Not at all sure
what this would be.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: binary at the front of CHANGES.txt
Posted by Yonik Seeley <yo...@apache.org>.
On 7/17/07, DM Smith <dm...@gmail.com> wrote:
> According to the UTF-8 spec \uFEFF is not a BOM. In UTF-8 the byte
> order is always the same.
But there is a BOM for UTF-8 (even though there is no endian
component, it does serve as a marker indicating the text file is
unicode text encoded in UTF-8).
http://unicode.org/faq/utf_bom.html#29
-Yonik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: binary at the front of CHANGES.txt
Posted by DM Smith <dm...@gmail.com>.
According to the UTF-8 spec \uFEFF is not a BOM. In UTF-8 the byte
order is always the same.
UTF-16 defines a BOM.
If it is a BOM, I bet it was put in there by an MS editor.
On Jul 17, 2007, at 1:58 PM, Yonik Seeley wrote:
> On 7/17/07, Steven Parkes <st...@esseff.org> wrote:
>> Can we get rid of the binary characters at the front of
>> CHANGES.txt? Or
>> do the mean something?
>
> AFAIK, It's a UTF-8 BOM. I don't know who's editor put it there, but
> it looks like the last time I edited CHANGES.txt it mangled it. I
> just removed it.
>
> -Yonik
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: binary at the front of CHANGES.txt
Posted by Yonik Seeley <yo...@apache.org>.
On 7/17/07, Steven Parkes <st...@esseff.org> wrote:
> Can we get rid of the binary characters at the front of CHANGES.txt? Or
> do the mean something?
AFAIK, It's a UTF-8 BOM. I don't know who's editor put it there, but
it looks like the last time I edited CHANGES.txt it mangled it. I
just removed it.
-Yonik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: binary at the front of CHANGES.txt
Posted by Chris Hostetter <ho...@fucit.org>.
: Can we get rid of the binary characters at the front of CHANGES.txt? Or
: do the mean something?
i may be wrong (I frequently am) but i think that's signifies (to
someone) that it's a UTF-8 file...
http://www.firstobject.com/dn_markutf8preamble.htm
...assuming that source can be trusted, it's a non-standard, not widely
supported, preamble that should be avoided.
personally, i don't think most of our files really need to be UTF-8. i'm
perfectly fine with removing it, but it's not going to do any good if
people's editors keep adding it back in.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org