extended ASCII character handling in JMeter/Java

[Sonam Chauhan <so...@ce.com.au>]

Hello  - I got hit by an old issue again this year, so wanted to ask about JMeter/Java handling of extended ASCII characters.

I have some testcase that use extended ASCII characters 252 and 255 ('ý' and 'ü') as record separators in text data posted by the JMeter HTTP sampler. The testcases were created on Windows XP - the data was simply copy/pasted into the JMeter GUI. 

When these tests ran on Linux, I found the LANG environment variable had to be set as follows to make the tests work  (email below from last year)
LANG=en_AU
This year, I moved to AIX this year and the tests failed again - the cause was 'ý' and 'ü' characters in the data were now posted in as a "?". I found the LANG variable on AIX was 'en_AU.utf8'. When set back to 'en_AU', the tests began posting in the correct values. 

My question is what is causing this behavior - does Java or JMeter use data in JMX files differently depending on the LANG variable in UNIX? 

Kind regards,
Sonam Chauhan
-- 
Corporate Express Australia Ltd. 
Phone: +61-2-93350725, Email: sonam.chauhan@ce.com.au

_____________________________________________
From: Sonam Chauhan 
Sent: Wednesday, 20 December 2006 2:27 PM
To: 'JMeter Users List'
Subject: JMeter under cron

Just a cautionary tale of running JMeter through a cron job on a Linux system.

We have a JMeter-based regression-test suite at work. This has run nightly for several years as a cron job. Recently, we added tests that post in extended ASCII data (which has 'ý' and 'ü' record separators) which sometimes passed, and sometimes failed. After much debugging I found the new tests failed when automatically run by cron, but passed when run by an interactive terminal session. 

When executed in an interactive terminal session, LANG is set to:
	LANG=en_AU
However, cron sets the Unix LANG environment variable to POSIX. Ie: 
LANG=POSIX
This seems to be causing the proble,.

I got the tests running by prefixing the test suite crontab entry with "export LANG=en_AU ;" 
ie: The entry is now: 
30 20 * * * export LANG=en_AU ; $HOME/runsuite.sh >> $HOME/tmp.out 2>&1
This got these tests running. 

Regards,
Sonam Chauhan

PS: 'locale -a' on the system shows that UTF-8 encoded English is also a support LANG attribute:  
en_AU.utf8
I guess this may be more pertinent for those whose testcases post in binary data.

Re: extended ASCII character handling in JMeter/Java

[sebb <se...@gmail.com>]

Which version of JMeter?

There have been quite a lot of recent changes in the handling of JMX files.

They now use UTF-8 by default - _file_encoding property in
saveservice.properties.

Originally JMeter used the platform default encoding, which would tend
to cause problems when the JMX file are used on different systems.

This was addressed in bug 36755, which was fixed in 2.3RC3.

There have also been various other encoding fixes - see the changes file.

How are you setting up the POST data?
Using files, or parameters?
What encoding are you using for the HTTP request?

What is the value of the Java property "file.encoding" on the various systems?
This is now shown in the jmeter log file.

On 23/10/2007, Sonam Chauhan <so...@ce.com.au> wrote:
> Hello  - I got hit by an old issue again this year, so wanted to ask about JMeter/Java handling of extended ASCII characters.
>
> I have some testcase that use extended ASCII characters 252 and 255 ('ı' and 'ü') as record separators in text data posted by the JMeter HTTP sampler. The testcases were created on Windows XP - the data was simply copy/pasted into the JMeter GUI.
>
> When these tests ran on Linux, I found the LANG environment variable had to be set as follows to make the tests work  (email below from last year)
> LANG=en_AU
> This year, I moved to AIX this year and the tests failed again - the cause was 'ı' and 'ü' characters in the data were now posted in as a "?". I found the LANG variable on AIX was 'en_AU.utf8'. When set back to 'en_AU', the tests began posting in the correct values.
>
> My question is what is causing this behavior - does Java or JMeter use data in JMX files differently depending on the LANG variable in UNIX?
>
> Kind regards,
> Sonam Chauhan
> --
> Corporate Express Australia Ltd.
> Phone: +61-2-93350725, Email: sonam.chauhan@ce.com.au
>
> _____________________________________________
> From: Sonam Chauhan
> Sent: Wednesday, 20 December 2006 2:27 PM
> To: 'JMeter Users List'
> Subject: JMeter under cron
>
> Just a cautionary tale of running JMeter through a cron job on a Linux system.
>
> We have a JMeter-based regression-test suite at work. This has run nightly for several years as a cron job. Recently, we added tests that post in extended ASCII data (which has 'ı' and 'ü' record separators) which sometimes passed, and sometimes failed. After much debugging I found the new tests failed when automatically run by cron, but passed when run by an interactive terminal session.
>
> When executed in an interactive terminal session, LANG is set to:
>        LANG=en_AU
> However, cron sets the Unix LANG environment variable to POSIX. Ie:
> LANG=POSIX
> This seems to be causing the proble,.
>
> I got the tests running by prefixing the test suite crontab entry with "export LANG=en_AU ;"
> ie: The entry is now:
> 30 20 * * * export LANG=en_AU ; $HOME/runsuite.sh >> $HOME/tmp.out 2>&1
> This got these tests running.
>
> Regards,
> Sonam Chauhan
>
> PS: 'locale -a' on the system shows that UTF-8 encoded English is also a support LANG attribute:
> en_AU.utf8
> I guess this may be more pertinent for those whose testcases post in binary data.
>
>
>