You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Cyril SCETBON <cy...@free.fr> on 2006/12/27 16:56:50 UTC
Weird behaviour with strings and accents
I'm using Modperl 2.0.2 with apache 2.0.55 and modperl is doing
something wrong with my string :
my $var="à présent protégé";
warn Dumper [$var];
my $var="à présent protégé".$login;
warn Dumper [$var];
$VAR1 = [
'à présent protégé'
];
$VAR1 = [
"\x{c3}\x{a0} pr\x{c3}\x{a9}sent
prot\x{c3}\x{a9}g\x{c3}\x{a9}login774"
];
__END__;
The accents are no more printed correctly !!
Any idea ?
Re: Weird behaviour with strings and accents
Posted by Marius Feraru <al...@n0i.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
tomas@tuxteam.de wrote:
> On Thu, Dec 28, 2006 at 07:18:29PM +0100, Cyril SCETBON wrote:
>> tomas@tuxteam.de wrote:
>>> On Wed, Dec 27, 2006 at 04:56:50PM +0100, Cyril SCETBON wrote:
>> [no utf8]
>
>> $VAR1 = [
>> 'à présent protégé'
>> ];
>> $VAR1 = [
>> "\x{c3}\x{a0} pr\x{c3}\x{a9}sent
>> prot\x{c3}\x{a9}g\x{c3}\x{a9}login774"
>> ];
>> It's really weird, isn't it ???
> No, it isn't. So your source is actually in iso-8859-1. Just the second
> half looks weird to me (what is changing the perceived encoding of the
> string after appending something seemingly harmless?
Not really, as Cyril didn't provide us with details about his
environment, so my best guess is that "$login" is already utf8 decoded,
that's why the "automatic decoding" happens at concatenating those strings.
- --
Marius Feraru
-----BEGIN PGP SIGNATURE-----
iD8DBQFFlS+VtZHp/AYZiNkRAo3rAJ9gdao+NjCGZVc55atRDvvRgOv+iwCgtl3X
F9TezyuFsOhak2bw0oXTK+s=
=bcsM
-----END PGP SIGNATURE-----
Re: Weird behaviour with strings and accents
Posted by to...@tuxteam.de.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Thu, Dec 28, 2006 at 07:18:29PM +0100, Cyril SCETBON wrote:
>
>
> tomas@tuxteam.de wrote:
> >-----BEGIN PGP SIGNED MESSAGE-----
> >Hash: SHA1
> >
> >On Wed, Dec 27, 2006 at 04:56:50PM +0100, Cyril SCETBON wrote:
[...]
> Here you can see the results when I use pragma utf8 :
>
> [use utf8]
>
> $VAR1 = [
> "\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}"
> ];
> $VAR1 = [
> "\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}login774"
> ];
Dang! :-)
> [no utf8]
>
> $VAR1 = [
> 'à présent protégé'
> ];
> $VAR1 = [
> "\x{c3}\x{a0} pr\x{c3}\x{a9}sent
> prot\x{c3}\x{a9}g\x{c3}\x{a9}login774"
> ];
>
> It's really weird, isn't it ???
No, it isn't. So your source is actually in iso-8859-1. Just the second
half looks weird to me (what is changing the perceived encoding of the
string after appending something seemingly harmless?
Anyway -- I'd suggest to stick to utf-8 (and use a text editor that
supports it). In the long run you'll have less headaches. In the
meantime, if your editor doesn't help you, you might try
use Encode 'decode';
...
$str = decode("iso-8859-1", 'à présent protégé');
(see man perluniintro, man perlunicode for the gory details).
HTH
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
iD8DBQFFlPyEBcgs9XrR2kYRAq/IAJ0WZ+oh8i8PHqzpKmcO1Ez+Tia91wCeJHhA
+brWNcpPEkPrSQBbm7t2Vrk=
=oq8I
-----END PGP SIGNATURE-----
Re: Weird behaviour with strings and accents
Posted by Lionel MARTIN <ma...@hotmail.com>.
Hi,
Having $VAR1 = ["\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}"] or $VAR1 =
["\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}login774"];
is really what you were expecting, i.e. true latin1 one byte characters. (it
appears that your dump shows the hexa code of non ASCII chars, i.e. with hex
code above \x{a0}, but that's the same).
On the other hand, it appears that when you are not doing "use UTF8", then,
everytime you are handling a string, it is turned into UTF8, like your "[no
utf8]" section shows. (perhaps because your source code is written in UTF8?)
I don't know why, but anyway, it appears that [use uft8] solves your
problem.
Lionel.
----- Original Message -----
From: "Cyril SCETBON" <cy...@free.fr>
To: <to...@tuxteam.de>
Cc: <mo...@perl.apache.org>
Sent: Thursday, December 28, 2006 7:18 PM
Subject: Re: Weird behaviour with strings and accents
>
>
> tomas@tuxteam.de wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On Wed, Dec 27, 2006 at 04:56:50PM +0100, Cyril SCETBON wrote:
>>> I'm using Modperl 2.0.2 with apache 2.0.55 and modperl is doing
>>> something wrong with my string :
>>>
>>> my $var="à présent protégé";
>>> warn Dumper [$var];
>>> my $var="à présent protégé".$login;
>>> warn Dumper [$var];
>>>
>>> $VAR1 = [
>>> 'à présent protégé'
>>> ];
>>> $VAR1 = [
>>> "\x{c3}\x{a0} pr\x{c3}\x{a9}sent
>>> prot\x{c3}\x{a9}g\x{c3}\x{a9}login774"
>>> ];
>>>
>>> __END__;
>>>
>>> The accents are no more printed correctly !!
>>
>> Hm. So it seems that Perl changes its notion of string encoding when
>> modifying it. Curious.
>>
>> Your source seems to be UTF-8. Have you tried to invoke the pragma "use
>> utf-8"?
> Here you can see the results when I use pragma utf8 :
>
> [use utf8]
>
> $VAR1 = [
> "\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}"
> ];
> $VAR1 = [
> "\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}login774"
> ];
>
> [no utf8]
>
> $VAR1 = [
> 'à présent protégé'
> ];
> $VAR1 = [
> "\x{c3}\x{a0} pr\x{c3}\x{a9}sent
> prot\x{c3}\x{a9}g\x{c3}\x{a9}login774"
> ];
>
> It's really weird, isn't it ???
>
>>
>> Regards
>> - -- tomás
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.1 (GNU/Linux)
>>
>> iD8DBQFFk1WbBcgs9XrR2kYRAuWUAJ4jOouhO0F5SjMIlRQFRZl5aGIGUwCfSi4Y
>> 9wzCCGflvOVt4nd8M7bVNjI=
>> =AAPm
>> -----END PGP SIGNATURE-----
>>
>>
>>
>
>
Re: Weird behaviour with strings and accents
Posted by Cyril SCETBON <cy...@free.fr>.
tomas@tuxteam.de wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Wed, Dec 27, 2006 at 04:56:50PM +0100, Cyril SCETBON wrote:
>> I'm using Modperl 2.0.2 with apache 2.0.55 and modperl is doing
>> something wrong with my string :
>>
>> my $var="à présent protégé";
>> warn Dumper [$var];
>> my $var="à présent protégé".$login;
>> warn Dumper [$var];
>>
>> $VAR1 = [
>> 'à présent protégé'
>> ];
>> $VAR1 = [
>> "\x{c3}\x{a0} pr\x{c3}\x{a9}sent
>> prot\x{c3}\x{a9}g\x{c3}\x{a9}login774"
>> ];
>>
>> __END__;
>>
>> The accents are no more printed correctly !!
>
> Hm. So it seems that Perl changes its notion of string encoding when
> modifying it. Curious.
>
> Your source seems to be UTF-8. Have you tried to invoke the pragma "use
> utf-8"?
Here you can see the results when I use pragma utf8 :
[use utf8]
$VAR1 = [
"\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}"
];
$VAR1 = [
"\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}login774"
];
[no utf8]
$VAR1 = [
'à présent protégé'
];
$VAR1 = [
"\x{c3}\x{a0} pr\x{c3}\x{a9}sent
prot\x{c3}\x{a9}g\x{c3}\x{a9}login774"
];
It's really weird, isn't it ???
>
> Regards
> - -- tomás
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
>
> iD8DBQFFk1WbBcgs9XrR2kYRAuWUAJ4jOouhO0F5SjMIlRQFRZl5aGIGUwCfSi4Y
> 9wzCCGflvOVt4nd8M7bVNjI=
> =AAPm
> -----END PGP SIGNATURE-----
>
>
>
Re: Weird behaviour with strings and accents
Posted by to...@tuxteam.de.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Wed, Dec 27, 2006 at 04:56:50PM +0100, Cyril SCETBON wrote:
> I'm using Modperl 2.0.2 with apache 2.0.55 and modperl is doing
> something wrong with my string :
>
> my $var="à présent protégé";
> warn Dumper [$var];
> my $var="à présent protégé".$login;
> warn Dumper [$var];
>
> $VAR1 = [
> 'à présent protégé'
> ];
> $VAR1 = [
> "\x{c3}\x{a0} pr\x{c3}\x{a9}sent
> prot\x{c3}\x{a9}g\x{c3}\x{a9}login774"
> ];
>
> __END__;
>
> The accents are no more printed correctly !!
Hm. So it seems that Perl changes its notion of string encoding when
modifying it. Curious.
Your source seems to be UTF-8. Have you tried to invoke the pragma "use
utf-8"?
Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
iD8DBQFFk1WbBcgs9XrR2kYRAuWUAJ4jOouhO0F5SjMIlRQFRZl5aGIGUwCfSi4Y
9wzCCGflvOVt4nd8M7bVNjI=
=AAPm
-----END PGP SIGNATURE-----