Bring the new I18NMatcher in line with LocaleAction [Bug 30046]

[Vadim Gritsenko <va...@reverycodes.com>]

Upayavira wrote:

> bugzilla@apache.org wrote:
> 
>> <http://issues.apache.org/bugzilla/show_bug.cgi?id=30046>.
 >>
>> ------- Additional Comments From vgritsenko@apache.org  2004-08-09 
>> 14:34 -------
>> Added code for checking locale stored in session and in cookie. Full 
>> locale
>> checking order now is:
>> * request parameter
>> * session attribute
>> * cookie
>> * sitemap parameter
>> * request locale(s) - optional, depends on use-locale(s) config 
>> parameters.
>> * default
>> * blank
>>
>> LocaleAction supports same order but has no sitemap parameter, and has no
>> default locale, and blank locale (because request locale checking is not
>> optional - and will always return some non null locale).
>>
>> Now, about setting locale into the request/session/cookie: this is not 
>> as simple
>> as it looks. Problem is that LocaleAction selects first available 
>> locale, but
>> I18nMatcher goes through *all* available locales till it finds matching
>> resource. Thus, for one resource it can result in one locale, and for 
>> other
>> resource - in other locale, so locale returned by I18nMatcher can not 
>> be used
>> for setting into session/cookie.
>>
>> I18nMatcher can be changed to be two-step procedure, (1) select 
>> locale, (2)
>> using selected locale search for resource in this locale (with variant ->
>> country -> language -> blank degradation). This modified I18nMatcher 
>> will be
>> more compatible with LocaleAction, and then it can be modified to set 
>> chosen
>> locale into session/cookie.
>>  
>>
> (on train - can't reply through Bugzilla)

It's Ok as long as you reply :-)


> I don't quite understand how your two stage procedure would work. As the 
> selection of the locale is fundamentally connected with the available 
> resources.

I understand what you mean.


> As you work through each method for selecting a locale, you 
> look for a resource using that method, if found, you exit. Otherwise, 
> you try the next method, and if necessary, you degrade from 
> variant/country/language, etc. How could this be done as separate select 
> locale/select resource stages?

The problem I'm trying to communicate is disconnect between LocaleAction 
and I18nMatcher. LocaleAction chooses a locale with disregard to the 
availability of the resources. Originally I thought that I18nMatcher is 
a nice shortcut combining LocaleAction and resource selection:

   <map:act type="locale">
     <map:select type="resource-exist">
       <map:when test="{../1}_{language}_{country}-{variant}.xml">
         <map:generate src="{../1}_{language}_{country}-{variant}.xml"/>
       </map:when>
       <map:when test="{../1}_{language}_{country}.xml">
         <map:generate src="{../1}_{language}_{country}.xml"/>
       </map:when>
       <map:when test="{../1}_{language}.xml">
         <map:generate src="{../1}_{language}.xml"/>
       </map:when>
       <map:otherwise>
         <map:generate src="{../1}.xml"/>
       </map:otherwise>
     </map:select>
   </map:act>

As you see, here locale selection step and resource selection steps are 
well separated: first you select locale, and only for this locale you go 
and try to locate a matching resource using (up to) 3 variations of this 
locale.

In I18nMatcher as it is implemented now, it goes through all locales, 
and within each locale, through three variations of locale, and tries to 
locate a matching resource. Thus, due to this difference in 
implementation, following is not possible:

  * Implementing "set locale" functionality in the I18nMatcher.
    (because request for another resource can result in another locale)
  * Mix and match LocaleAction and I18nMatcher in same application.
    (LocaleAction remembers selected locale, I18nMatcher does not. And,
     selection algorithm does not match)

What I would like to do is to come up with approach which will unify 
these two together, by making I18nMatcher closer to LocaleAction, or, if 
you have suggestion, by enhancing LocaleAction.


However, I have trouble understanding how I18nMatcher suppose to work in 
practice when it mixes different locales for the same user. Let's go 
through simple use case:
  * Browser has configuration:
     Accept-Language: en, fr
  * Site has resources:
     image/en/a.gif
     image/fr/a.gif
     image/fr/b.gif
     image/b.gif
  * Site has page:
     <html><img src="a.gif"/><img src="b.gif"/></html>

Now, question: What would you expect to see? What would your graphics 
designer expect to see (especially if he uses text - in - graphics)?

Vadim

Re: Bring the new I18NMatcher in line with LocaleAction [Bug 30046]

[Vadim Gritsenko <va...@reverycodes.com>]

Upayavira wrote:

> Vadim Gritsenko wrote:
> 
>> Ok, I (majorly) reworked both LocaleAction and I18nMatcher.
>>
>> Also, I removed "language-attribute", "country-attribute", 
>> "variant-attribute" configuration elements from LocaleAction - IMHO 
>> they are clear FS - what's the point in customizing sitemap variable 
>> names. Language sitemap variable renamed to "language", same as in 
>> I18nMatcher.
> 
> 
> Fair enough. Although it is a feature removal.

Is it (necessarily) bad? Mentioning in the "Update" doc should be enough.


>> Please take a look, check that everything is ok :-)
>>
>> I was thinking: may be I18nMatcher should be renamed to LocaleMatcher?
> 
> 
> I had thought of that myself. I'm okay with that too.

I'll go then with renaming it and merging it into the 2.1 branch.

Vadim

Re: Bring the new I18NMatcher in line with LocaleAction [Bug 30046]

[Upayavira <uv...@upaya.co.uk>]

Vadim Gritsenko wrote:

> Upayavira wrote:
>
>> Fine by me too!
>
> Ok, I (majorly) reworked both LocaleAction and I18nMatcher. I also 
> noticed that original I18nMatcher was not properly handling locales 
> with countries and variants (AFAIU, constructor new Locale(locale, "") 
> as first argument takes language and not locale string).

Great. I'm no I18n expert, so these are details I wasn't able to take 
care of.

> Also, I removed "language-attribute", "country-attribute", 
> "variant-attribute" configuration elements from LocaleAction - IMHO 
> they are clear FS - what's the point in customizing sitemap variable 
> names. Language sitemap variable renamed to "language", same as in 
> I18nMatcher.

Fair enough. Although it is a feature removal.

> Please take a look, check that everything is ok :-)
>
> I was thinking: may be I18nMatcher should be renamed to LocaleMatcher?

I had thought of that myself. I'm okay with that too.

Regards, Upayavira

Re: Bring the new I18NMatcher in line with LocaleAction [Bug 30046]

[Vadim Gritsenko <va...@reverycodes.com>]

Upayavira wrote:

> Fine by me too!

Ok, I (majorly) reworked both LocaleAction and I18nMatcher. I also 
noticed that original I18nMatcher was not properly handling locales with 
countries and variants (AFAIU, constructor new Locale(locale, "") as 
first argument takes language and not locale string).

Also, I removed "language-attribute", "country-attribute", 
"variant-attribute" configuration elements from LocaleAction - IMHO they 
are clear FS - what's the point in customizing sitemap variable names. 
Language sitemap variable renamed to "language", same as in I18nMatcher.

Please take a look, check that everything is ok :-)

I was thinking: may be I18nMatcher should be renamed to LocaleMatcher?

Vadim

Re: Bring the new I18NMatcher in line with LocaleAction [Bug 30046]

[Upayavira <uv...@upaya.co.uk>]

Vadim Gritsenko wrote:

> Upayavira wrote:
>
>> Vadim Gritsenko wrote:
>>
>>>
>>> What I would like to do is to come up with approach which will unify 
>>> these two together, by making I18nMatcher closer to LocaleAction, 
>>> or, if you have suggestion, by enhancing LocaleAction.
>>
>>
>> It would be good to see them unified, but I can't yet see how - as, 
>> at least at the moment, their use cases are quite different. And at 
>> the mo, I'm okay with the difference.
>
> >
>
>> The thing is, I have a site that has maybe 100 pages. We are trying 
>> to get people to translate it - we've got Polish now, Chinese and 
>> Hindi are probably on the way (why not French and German first :-(   
>> ), but the translators will not translate everything. So the site has 
>> to be able to gracefully downgrade to the next best language when a 
>> page hasn't been translated (in most cases English).
>
>
> Hmmm. Idea: How about implementing advanced mode for LocaleAction 
> which would negotiate locale based on client request and i18n 
> catalogue you have? This would bring it step closer to I18nMatcher. :-)
>
Sounds fine :-)

>
>> I guess we could add a configuration parameter to the definition of 
>> the matcher, to change the way it searches for sources, and to store 
>> the result in the session, for compatibility with LocaleAction. I'd 
>> be okay with that, so long as the other approach is available too!
>
>
> Ok, let's then implement two modes:
>  (a) Same as in LocaleAction.
>  (b) Same as now.
> And, optionally, results can be saved into session/cookie/etc.

Fine by me too!

If it helps to make them work more similarly, I'd be happy for the
current behaviour of the i18nmatcher to be the 'advanced' mode, as it
would be on the locale action. Does that help?

Regards, Upayavira

Re: Bring the new I18NMatcher in line with LocaleAction [Bug 30046]

[Vadim Gritsenko <va...@reverycodes.com>]

Upayavira wrote:

> Vadim Gritsenko wrote:
>>
>> What I would like to do is to come up with approach which will unify 
>> these two together, by making I18nMatcher closer to LocaleAction, or, 
>> if you have suggestion, by enhancing LocaleAction.
> 
> It would be good to see them unified, but I can't yet see how - as, at 
> least at the moment, their use cases are quite different. And at the mo, 
> I'm okay with the difference.
 >
> The thing is, I have a site that has maybe 100 pages. We are trying to 
> get people to translate it - we've got Polish now, Chinese and Hindi are 
> probably on the way (why not French and German first :-(   ), but the 
> translators will not translate everything. So the site has to be able to 
> gracefully downgrade to the next best language when a page hasn't been 
> translated (in most cases English).

Hmmm. Idea: How about implementing advanced mode for LocaleAction which 
would negotiate locale based on client request and i18n catalogue you 
have? This would bring it step closer to I18nMatcher. :-)


> I guess we could add a configuration parameter to the definition of the 
> matcher, to change the way it searches for sources, and to store the 
> result in the session, for compatibility with LocaleAction. I'd be okay 
> with that, so long as the other approach is available too!

Ok, let's then implement two modes:
  (a) Same as in LocaleAction.
  (b) Same as now.
And, optionally, results can be saved into session/cookie/etc.

Vadim

Re: Bring the new I18NMatcher in line with LocaleAction [Bug 30046]

[Upayavira <uv...@upaya.co.uk>]

Vadim Gritsenko wrote:

>> I don't quite understand how your two stage procedure would work. As 
>> the selection of the locale is fundamentally connected with the 
>> available resources.
>
> I understand what you mean.
>
>> As you work through each method for selecting a locale, you look for 
>> a resource using that method, if found, you exit. Otherwise, you try 
>> the next method, and if necessary, you degrade from 
>> variant/country/language, etc. How could this be done as separate 
>> select locale/select resource stages?
>
> The problem I'm trying to communicate is disconnect between 
> LocaleAction and I18nMatcher. LocaleAction chooses a locale with 
> disregard to the availability of the resources. Originally I thought 
> that I18nMatcher is a nice shortcut combining LocaleAction and 
> resource selection:
>
>   <map:act type="locale">
>     <map:select type="resource-exist">
>       <map:when test="{../1}_{language}_{country}-{variant}.xml">
>         <map:generate src="{../1}_{language}_{country}-{variant}.xml"/>
>       </map:when>
>       <map:when test="{../1}_{language}_{country}.xml">
>         <map:generate src="{../1}_{language}_{country}.xml"/>
>       </map:when>
>       <map:when test="{../1}_{language}.xml">
>         <map:generate src="{../1}_{language}.xml"/>
>       </map:when>
>       <map:otherwise>
>         <map:generate src="{../1}.xml"/>
>       </map:otherwise>
>     </map:select>
>   </map:act>
>
> As you see, here locale selection step and resource selection steps 
> are well separated: first you select locale, and only for this locale 
> you go and try to locate a matching resource using (up to) 3 
> variations of this locale.
>
> In I18nMatcher as it is implemented now, it goes through all locales, 
> and within each locale, through three variations of locale, and tries 
> to locate a matching resource. Thus, due to this difference in 
> implementation, following is not possible:
>
>  * Implementing "set locale" functionality in the I18nMatcher.
>    (because request for another resource can result in another locale)
>  * Mix and match LocaleAction and I18nMatcher in same application.
>    (LocaleAction remembers selected locale, I18nMatcher does not. And,
>     selection algorithm does not match)

Correct. The LocaleAction, in my mind, is good for sites where you have 
_everything_ translated into all of your languages. The I18nMatcher 
handles situations (as does httpd) where you don't necessarily.

> What I would like to do is to come up with approach which will unify 
> these two together, by making I18nMatcher closer to LocaleAction, or, 
> if you have suggestion, by enhancing LocaleAction.

It would be good to see them unified, but I can't yet see how - as, at 
least at the moment, their use cases are quite different. And at the mo, 
I'm okay with the difference.

> However, I have trouble understanding how I18nMatcher suppose to work 
> in practice when it mixes different locales for the same user. Let's 
> go through simple use case:
>  * Browser has configuration:
>     Accept-Language: en, fr
>  * Site has resources:
>     image/en/a.gif
>     image/fr/a.gif
>     image/fr/b.gif
>     image/b.gif
>  * Site has page:
>     <html><img src="a.gif"/><img src="b.gif"/></html>

Well, I wouldn't use it like that! I'd do:
<map:match pattern="source/*/foo.xml">
  <map:generate src="{source}"/>
  <map:transform src="add-images-to-your-page.xsl">
    <map:parameter name="locale" value="{locale}"/>
  </map:transform>
  <map:serialize type="html"/>
</map:match>

So you use the locale found by the i18n matcher to view resources on 
that page.

> Now, question: What would you expect to see? What would your graphics 
> designer expect to see (especially if he uses text - in - graphics)?

The thing is, I have a site that has maybe 100 pages. We are trying to 
get people to translate it - we've got Polish now, Chinese and Hindi are 
probably on the way (why not French and German first :-(   ), but the 
translators will not translate everything. So the site has to be able to 
gracefully downgrade to the next best language when a page hasn't been 
translated (in most cases English). Actually, ideally, in such a 
situation, it would say, e.g. in Polish "Sorry, this page wasn't 
available in Polish. Here is the English version instead".

I guess we could add a configuration parameter to the definition of the 
matcher, to change the way it searches for sources, and to store the 
result in the session, for compatibility with LocaleAction. I'd be okay 
with that, so long as the other approach is available too!

Can you say more about your concerns and how the i18nmatcher doesn't 
work for you?

Regards, Upayavira