You are viewing a plain text version of this content. The canonical link for it is here.
Posted to fop-users@xmlgraphics.apache.org by Brad Smith <us...@gmail.com> on 2007/06/05 14:39:27 UTC
Korean words breaking in the middle
I am having an issue with Korean words not breaking properly. I'm
using a fop build based on the svn trunk as of 2007-05-01. The
attached screenshot shows an example, courtesy of our Korean
translator. The blue rectangle on the left shows the text as-rendered,
with each line ending in a mid-word wrap. The rectangle on the left
shows the complete words as they should be rendered (unlike Japanese
and Chinese, Korean uses space-delimited words like western
languages). I've checked the source and there are no spaces within
these words.
Any ideas? Is a fix in the works for this?
--Brad
Re: Korean words breaking in the middle
Posted by Brad Smith <us...@gmail.com>.
Hi Manuel,
I'm trying to rebuild fop with the tweaked linebreak rules as
described below, but I it fails, saying:
junit:
BUILD FAILED
/home/brad/Work/fop/trunk/build.xml:947: NOTE:
**************************************************************************
* One or more of the Junit tests had Failures or Errors or were skipped! *
* Please check the output above for relevant messages. *
* Or use the "junit-reports" target to generate HTML test reports. *
**************************************************************************
The "above output" is hundreds of lines of info and warning messages,
but no errors. I ran the junit-reports target and it shows 422 tests
with 0 errors. I'm not sure what "errors were skipped" would entail.
Any suggestions would be appreciated. I'm fairly certain I saw this
the last time I attempted a build, so I don't think it's due to the
change I made.
--Brad
On 1/2/02, Manuel Mall <ma...@apache.org> wrote:
> On Saturday 09 June 2007 00:50, Brad Smith wrote:
> > I'm afraid most of the implementation-level discussion of this is
> > above me, but I can give you a hint that might prove helpful: our
> > translator did some experimentation and found that fop 0.93 breaks
> > ko-KR correctly, whereas fop trunk does not. On the other hand, 0.93
> > does not correctly break japanese lines, but trunk does.
> >
>
> Brad,
>
> as stated before fop-trunk breaks Korean basically the same as Chinese,
> that is each consecutive pair of Korean characters are treated as a
> break opportunity. This is consistent with the default specified in the
> relevant Unicode standard.
>
> If you want to change this behaviour you need to change the line
> breaking pair table used by fop, this means building yourself a custom
> fop version.
>
> Here are some hints (not tested) how to do that.
>
> 1) Edit the file src/codegen/unicode/data/LineBreakPairTable.txt, which
> is basically a table, with a text editor. In the H2/H2, H2/H3, H3/H2,
> H3/H3 cells change the character _ (underscore) to % (percent). Note:
> Underscore indicates a direct break opportunity between characters of
> this class while percent indicates an indirect opportunity, that is
> break only if spaces intervene.
>
> 2) Now run the codegen-unicode ant target contained in the standard fop
> build.xml file. This should create a new LineBreakUtils.java file in
> the fop top level build directory.
>
> 3) Copy that file to src/java/org/apache/fop/text/linebreak
>
> 4) Rebuild fop
>
> Hope this helps
>
> Manuel
>
> > --Brad
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
> > For additional commands, e-mail:
> > fop-users-help@xmlgraphics.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
> For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
>
>
--
~ Second Shift: An original, serialized audio adventure ~
http://www.secondshiftpodcast.com
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Korean words breaking in the middle
Posted by Brad Smith <us...@gmail.com>.
I'm afraid most of the implementation-level discussion of this is
above me, but I can give you a hint that might prove helpful: our
translator did some experimentation and found that fop 0.93 breaks
ko-KR correctly, whereas fop trunk does not. On the other hand, 0.93
does not correctly break japanese lines, but trunk does.
--Brad
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Korean words breaking in the middle
Posted by Manuel Mall <ma...@apache.org>.
On Tuesday 05 June 2007 21:14, Manuel Mall wrote:
> On Tuesday 05 June 2007 20:39, Brad Smith wrote:
> > I am having an issue with Korean words not breaking properly. I'm
> > using a fop build based on the svn trunk as of 2007-05-01. The
> > attached screenshot shows an example, courtesy of our Korean
> > translator. The blue rectangle on the left shows the text
> > as-rendered, with each line ending in a mid-word wrap. The
> > rectangle on the left shows the complete words as they should be
> > rendered (unlike Japanese and Chinese, Korean uses space-delimited
> > words like western
> > languages). I've checked the source and there are no spaces within
> > these words.
> >
> > Any ideas? Is a fix in the works for this?
>
> Brad,
>
> fop uses a pair based line breaking algorithm based on Unicode
> Standard Annex #14 (http://www.unicode.org/reports/tr14/). This
> document actually makes quite a few comments on special cases for the
> Korean language. Most of that is straight above my head as I have no
> idea what for example a 'Hangul syllable and conjoining jamo'
> actually is. It would be really helpful if someone with real
> knowledge of the Korean language could read and interpret for us
> those parts of the standard and may be point out where exactly we may
> be doing something wrong. It sounds to me like Korean has two
> different ways of linebreaking and possibly our default UAX#14
> implementation may only support one.
>
> I know this doesn't help you directly but the first step here for me
> would be getting help in understanding the issue in the context of
> the Unicode standard annex we try to implement.
>
> Thanks
>
> Manuel
>
The following snippet is taken from the Unicode Standard Annex #14:
= START =
H2: Hangul LV Syllable (B/A)
This class includes all characters of Hangul Syllable Type LV.
Together with conjoining jamos, Hangul syllables form Korean Syllable
Blocks, which are kept together; see [Boundaries]. Korean uses
space-based line breaking in many styles of documents. To support
these, Hangul syllables and conjoining jamo need to be tailored to use
class AL, while the default in this specification is class ID, which
supports the case of Korean documents not using space-based line
breaking. See Section 8.1, Types of Tailoring. See also JL, JT, JV, and
H3.
= END =
This seems to indicate that the default is 'ideographic' linebreaking
for Korean and not space based linebreaking and FOP only implements the
default and does not support tailoring.
You could build yourself a custom fop version with a linebreaking table
which treats character class H2 like AL as suggested in the snippet
from the Unicode Annex above. I can give you pointers how to do that.
Of course the better solution would be to allow some tailoring of the
linebreaking code but I don't have the time to look into that atm. If
you would like to have a go feel most welcome to do so. Again, I would
be happy to give pointers.
Regards
Manuel
> > --Brad
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
> For additional commands, e-mail:
> fop-users-help@xmlgraphics.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Korean words breaking in the middle
Posted by "J.Pietschmann" <j3...@yahoo.de>.
Manuel Mall wrote:
> fop uses a pair based line breaking algorithm based on Unicode Standard
> Annex #14 (http://www.unicode.org/reports/tr14/). This document
> actually makes quite a few comments on special cases for the Korean
> language. Most of that is straight above my head as I have no idea what
> for example a 'Hangul syllable and conjoining jamo' actually is. It
> would be really helpful if someone with real knowledge of the Korean
> language could read and interpret for us those parts of the standard
> and may be point out where exactly we may be doing something wrong. It
> sounds to me like Korean has two different ways of linebreaking and
> possibly our default UAX#14 implementation may only support one.
IIRC I've set the line breaking opportunity to "break" for all
pairs of CJK code points, because the original Unicode tables
didn't provide a useful value (that's the fall back recommended
by the spec in case no better value is available). That's
simplistic. Maybe looking at pango and perhaps a variety of other
OSS text rendering projects can help. I didn't get enough spare
time to do it myself yet.
J.Pietschmann
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Korean words breaking in the middle
Posted by Manuel Mall <ma...@apache.org>.
On Tuesday 05 June 2007 20:39, Brad Smith wrote:
> I am having an issue with Korean words not breaking properly. I'm
> using a fop build based on the svn trunk as of 2007-05-01. The
> attached screenshot shows an example, courtesy of our Korean
> translator. The blue rectangle on the left shows the text
> as-rendered, with each line ending in a mid-word wrap. The rectangle
> on the left shows the complete words as they should be rendered
> (unlike Japanese and Chinese, Korean uses space-delimited words like
> western
> languages). I've checked the source and there are no spaces within
> these words.
>
> Any ideas? Is a fix in the works for this?
>
Brad,
fop uses a pair based line breaking algorithm based on Unicode Standard
Annex #14 (http://www.unicode.org/reports/tr14/). This document
actually makes quite a few comments on special cases for the Korean
language. Most of that is straight above my head as I have no idea what
for example a 'Hangul syllable and conjoining jamo' actually is. It
would be really helpful if someone with real knowledge of the Korean
language could read and interpret for us those parts of the standard
and may be point out where exactly we may be doing something wrong. It
sounds to me like Korean has two different ways of linebreaking and
possibly our default UAX#14 implementation may only support one.
I know this doesn't help you directly but the first step here for me
would be getting help in understanding the issue in the context of the
Unicode standard annex we try to implement.
Thanks
Manuel
> --Brad
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org