CVS Dataset Config Usage?

[Knut Borchart <kb...@gmail.com>]

Hi,

i am experimenting with the csv dataset config but it seems like i do
not quite understand its inner workings. What i am trying to achieve
is the following:

- keep a file of usernames, passwords and other config for users
around which is always 100 lines long.
- for testing a new jmeter script i would like use different settings
for threads and repeat (eg. 2 threads, 10 repeats)

I thought that jmeter reads the cvs file at the beginning of the
testcase, assigns values to the different threads (=users) and thats
it. But that does not work. With 1 thread and 10 repeats the second
repeat of the test gets the config for user 2 (=line 2 of csv file).
So it seems like the file is read in every iteration.

How is the csv config thought to be used? Do i have to exactly match
the lines in the file to the number of threads i want to use? What if
one of the threads gets an error and is stopped? Will not the
correlation between thread <> csv config will be mixed up?

Kind regards, peter

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org

Re: CVS Dataset Config Usage?

[sebb <se...@gmail.com>]

I forgot to say that the file is currently shared between thread groups as well.

If you want the file to be used independently in different thread
groups, a work-round is to use a copy of the file, or use a different
name for the same file, e.g.

data.csv
./data.csv

On 09/08/07, sebb <se...@gmail.com> wrote:
> That's not true - the file is shared between threads.
>
> When a thread needs data from the file, it will get the next entry in
> the file, on a first come first served basis.
>
>
>
> On 31/07/07, Avishek Daga <av...@gmail.com> wrote:
> >
> > Each thread has its own copy of the csv file and iterates through the file
> > from the start
> >
> >
> >
> > Knut Borchart wrote:
> > >
> > > Hi,
> > >
> > > i am experimenting with the csv dataset config but it seems like i do
> > > not quite understand its inner workings. What i am trying to achieve
> > > is the following:
> > >
> > > - keep a file of usernames, passwords and other config for users
> > > around which is always 100 lines long.
> > > - for testing a new jmeter script i would like use different settings
> > > for threads and repeat (eg. 2 threads, 10 repeats)
> > >
> > > I thought that jmeter reads the cvs file at the beginning of the
> > > testcase, assigns values to the different threads (=users) and thats
> > > it. But that does not work. With 1 thread and 10 repeats the second
> > > repeat of the test gets the config for user 2 (=line 2 of csv file).
> > > So it seems like the file is read in every iteration.
> > >
> > > How is the csv config thought to be used? Do i have to exactly match
> > > the lines in the file to the number of threads i want to use? What if
> > > one of the threads gets an error and is stopped? Will not the
> > > correlation between thread <> csv config will be mixed up?
> > >
> > > Kind regards, peter
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> > >
> > >
> > >
> >
> > --
> > View this message in context: http://www.nabble.com/CVS-Dataset-Config-Usage--tf4116129.html#a11921316
> > Sent from the JMeter - User mailing list archive at Nabble.com.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> >
> >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org

Re: CVS Dataset Config Usage?

[sebb <se...@gmail.com>]

That's not true - the file is shared between threads.

When a thread needs data from the file, it will get the next entry in
the file, on a first come first served basis.



On 31/07/07, Avishek Daga <av...@gmail.com> wrote:
>
> Each thread has its own copy of the csv file and iterates through the file
> from the start
>
>
>
> Knut Borchart wrote:
> >
> > Hi,
> >
> > i am experimenting with the csv dataset config but it seems like i do
> > not quite understand its inner workings. What i am trying to achieve
> > is the following:
> >
> > - keep a file of usernames, passwords and other config for users
> > around which is always 100 lines long.
> > - for testing a new jmeter script i would like use different settings
> > for threads and repeat (eg. 2 threads, 10 repeats)
> >
> > I thought that jmeter reads the cvs file at the beginning of the
> > testcase, assigns values to the different threads (=users) and thats
> > it. But that does not work. With 1 thread and 10 repeats the second
> > repeat of the test gets the config for user 2 (=line 2 of csv file).
> > So it seems like the file is read in every iteration.
> >
> > How is the csv config thought to be used? Do i have to exactly match
> > the lines in the file to the number of threads i want to use? What if
> > one of the threads gets an error and is stopped? Will not the
> > correlation between thread <> csv config will be mixed up?
> >
> > Kind regards, peter
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> >
> >
> >
>
> --
> View this message in context: http://www.nabble.com/CVS-Dataset-Config-Usage--tf4116129.html#a11921316
> Sent from the JMeter - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org

RE: CVS Dataset Config Usage?

[iblavins <ib...@temenos.com>]

G'day

Not at this stage, I am not in a position to support it. (Plus I don't own
the intellectual property and now is not the time to be asking the
organisation I work for to release it.)

That being said I am hoping at the end of this contract to make some kind of
contribution to JMeter, probably on the supporting infrastructure side.



Ian Blavins
Contract Performance Engineer
Temenos


-----Original Message-----
From: Matt [mailto:mattmoran76@gmail.com] 
Sent: 31 July 2007 12:53
To: JMeter Users List
Subject: Re: CVS Dataset Config Usage?

Sounds cool, you not open sourcing it?

On 31/07/07, iblavins <ib...@temenos.com> wrote:
>
> G'day
>
> I had to overcome this.
>
> I ended up:
>
> - writing a data server to serve unique values from the data files to
> callers
> - adding a sampler step to the test plan to retrieve the data from the
> data
> server on each iteration
> - adding a regular expression step to the test plan to bring the data from
> the above call into a test plan variable
> - passing the variable above to the existing step that runs the sampler
> against the system under test
>
> The central server guarantees uniqueness by serving the rows sequentially
> to
> callers. For each file at start up there is a flag indicating whether the
> file can be rewound and served again at end of file. If not then
> attempting
> to read beyond the amount of data results in "<no more data>" or similar
> being returned to the data getter sampler.
>
> If you want to go this way you need to be sure that getting the data from
> the data server does not become the bottleneck in your test. To this end
> you
> should have the data server load the whole data file(s) into memory on
> start
> up.
>
> For very large tests you also need to be conscious of the number of
> concurrent threads that can be handled by the data server. I implemented a
> two level structure where there is a set of concentrators in front of the
> data server with thread pooling in both the data getter sampler and in the
> concentrator.
>
> You also need to exclude the time to retrieve the data from the response
> time reported by your test.
>
>
>
>
> Ian Blavins
> Contract Performance Engineer
> Temenos
>
> -----Original Message-----
> From: Avishek Daga [mailto:avishekdaga@gmail.com]
> Sent: 31 July 2007 11:00
> To: jmeter-user@jakarta.apache.org
> Subject: Re: CVS Dataset Config Usage?
>
>
> Each thread has its own copy of the csv file and iterates through the file
> from the start
>
>
>
> Knut Borchart wrote:
> >
> > Hi,
> >
> > i am experimenting with the csv dataset config but it seems like i do
> > not quite understand its inner workings. What i am trying to achieve
> > is the following:
> >
> > - keep a file of usernames, passwords and other config for users
> > around which is always 100 lines long.
> > - for testing a new jmeter script i would like use different settings
> > for threads and repeat (eg. 2 threads, 10 repeats)
> >
> > I thought that jmeter reads the cvs file at the beginning of the
> > testcase, assigns values to the different threads (=users) and thats
> > it. But that does not work. With 1 thread and 10 repeats the second
> > repeat of the test gets the config for user 2 (=line 2 of csv file).
> > So it seems like the file is read in every iteration.
> >
> > How is the csv config thought to be used? Do i have to exactly match
> > the lines in the file to the number of threads i want to use? What if
> > one of the threads gets an error and is stopped? Will not the
> > correlation between thread <> csv config will be mixed up?
> >
> > Kind regards, peter
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> >
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/CVS-Dataset-Config-Usage--tf4116129.html#a11921316
> Sent from the JMeter - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
>
>
> This email (and any attachments) contains confidential information, and is
> intended
> only for the named recipient. Distribution or copying of this email by
> anyone
> other than the named recipient is prohibited. If you are not the named or
> intended recipient, please notify TEMENOS or the sender immediately and
> permanently destroy this email (and any attachments) and all copies of it.
> No
> member of TEMENOS Group AG or any of its associated or affiliated
> companies is
> liable for any errors or omissions in the content or transmission of this
> email. Any opinions contained in this email are solely those of the author
> and,
> unless clearly indicated otherwise in writing, are not endorsed by any
> member
> of TEMENOS Group AG or any of its associated and affiliated companies.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>



This email (and any attachments) contains confidential information, and is intended 
only for the named recipient. Distribution or copying of this email by anyone 
other than the named recipient is prohibited. If you are not the named or 
intended recipient, please notify TEMENOS or the sender immediately and 
permanently destroy this email (and any attachments) and all copies of it. No 
member of TEMENOS Group AG or any of its associated or affiliated companies is 
liable for any errors or omissions in the content or transmission of this 
email. Any opinions contained in this email are solely those of the author and, 
unless clearly indicated otherwise in writing, are not endorsed by any member 
of TEMENOS Group AG or any of its associated and affiliated companies. 


---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org

Re: CVS Dataset Config Usage?

[Matt <ma...@gmail.com>]

Sounds cool, you not open sourcing it?

On 31/07/07, iblavins <ib...@temenos.com> wrote:
>
> G'day
>
> I had to overcome this.
>
> I ended up:
>
> - writing a data server to serve unique values from the data files to
> callers
> - adding a sampler step to the test plan to retrieve the data from the
> data
> server on each iteration
> - adding a regular expression step to the test plan to bring the data from
> the above call into a test plan variable
> - passing the variable above to the existing step that runs the sampler
> against the system under test
>
> The central server guarantees uniqueness by serving the rows sequentially
> to
> callers. For each file at start up there is a flag indicating whether the
> file can be rewound and served again at end of file. If not then
> attempting
> to read beyond the amount of data results in "<no more data>" or similar
> being returned to the data getter sampler.
>
> If you want to go this way you need to be sure that getting the data from
> the data server does not become the bottleneck in your test. To this end
> you
> should have the data server load the whole data file(s) into memory on
> start
> up.
>
> For very large tests you also need to be conscious of the number of
> concurrent threads that can be handled by the data server. I implemented a
> two level structure where there is a set of concentrators in front of the
> data server with thread pooling in both the data getter sampler and in the
> concentrator.
>
> You also need to exclude the time to retrieve the data from the response
> time reported by your test.
>
>
>
>
> Ian Blavins
> Contract Performance Engineer
> Temenos
>
> -----Original Message-----
> From: Avishek Daga [mailto:avishekdaga@gmail.com]
> Sent: 31 July 2007 11:00
> To: jmeter-user@jakarta.apache.org
> Subject: Re: CVS Dataset Config Usage?
>
>
> Each thread has its own copy of the csv file and iterates through the file
> from the start
>
>
>
> Knut Borchart wrote:
> >
> > Hi,
> >
> > i am experimenting with the csv dataset config but it seems like i do
> > not quite understand its inner workings. What i am trying to achieve
> > is the following:
> >
> > - keep a file of usernames, passwords and other config for users
> > around which is always 100 lines long.
> > - for testing a new jmeter script i would like use different settings
> > for threads and repeat (eg. 2 threads, 10 repeats)
> >
> > I thought that jmeter reads the cvs file at the beginning of the
> > testcase, assigns values to the different threads (=users) and thats
> > it. But that does not work. With 1 thread and 10 repeats the second
> > repeat of the test gets the config for user 2 (=line 2 of csv file).
> > So it seems like the file is read in every iteration.
> >
> > How is the csv config thought to be used? Do i have to exactly match
> > the lines in the file to the number of threads i want to use? What if
> > one of the threads gets an error and is stopped? Will not the
> > correlation between thread <> csv config will be mixed up?
> >
> > Kind regards, peter
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> >
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/CVS-Dataset-Config-Usage--tf4116129.html#a11921316
> Sent from the JMeter - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
>
>
> This email (and any attachments) contains confidential information, and is
> intended
> only for the named recipient. Distribution or copying of this email by
> anyone
> other than the named recipient is prohibited. If you are not the named or
> intended recipient, please notify TEMENOS or the sender immediately and
> permanently destroy this email (and any attachments) and all copies of it.
> No
> member of TEMENOS Group AG or any of its associated or affiliated
> companies is
> liable for any errors or omissions in the content or transmission of this
> email. Any opinions contained in this email are solely those of the author
> and,
> unless clearly indicated otherwise in writing, are not endorsed by any
> member
> of TEMENOS Group AG or any of its associated and affiliated companies.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>

RE: CVS Dataset Config Usage?

[iblavins <ib...@temenos.com>]

G'day

I had to overcome this. 

I ended up:

- writing a data server to serve unique values from the data files to
callers
- adding a sampler step to the test plan to retrieve the data from the data
server on each iteration
- adding a regular expression step to the test plan to bring the data from
the above call into a test plan variable
- passing the variable above to the existing step that runs the sampler
against the system under test

The central server guarantees uniqueness by serving the rows sequentially to
callers. For each file at start up there is a flag indicating whether the
file can be rewound and served again at end of file. If not then attempting
to read beyond the amount of data results in "<no more data>" or similar
being returned to the data getter sampler.

If you want to go this way you need to be sure that getting the data from
the data server does not become the bottleneck in your test. To this end you
should have the data server load the whole data file(s) into memory on start
up. 

For very large tests you also need to be conscious of the number of
concurrent threads that can be handled by the data server. I implemented a
two level structure where there is a set of concentrators in front of the
data server with thread pooling in both the data getter sampler and in the
concentrator.

You also need to exclude the time to retrieve the data from the response
time reported by your test.




Ian Blavins
Contract Performance Engineer
Temenos

-----Original Message-----
From: Avishek Daga [mailto:avishekdaga@gmail.com] 
Sent: 31 July 2007 11:00
To: jmeter-user@jakarta.apache.org
Subject: Re: CVS Dataset Config Usage?


Each thread has its own copy of the csv file and iterates through the file
from the start



Knut Borchart wrote:
> 
> Hi,
> 
> i am experimenting with the csv dataset config but it seems like i do
> not quite understand its inner workings. What i am trying to achieve
> is the following:
> 
> - keep a file of usernames, passwords and other config for users
> around which is always 100 lines long.
> - for testing a new jmeter script i would like use different settings
> for threads and repeat (eg. 2 threads, 10 repeats)
> 
> I thought that jmeter reads the cvs file at the beginning of the
> testcase, assigns values to the different threads (=users) and thats
> it. But that does not work. With 1 thread and 10 repeats the second
> repeat of the test gets the config for user 2 (=line 2 of csv file).
> So it seems like the file is read in every iteration.
> 
> How is the csv config thought to be used? Do i have to exactly match
> the lines in the file to the number of threads i want to use? What if
> one of the threads gets an error and is stopped? Will not the
> correlation between thread <> csv config will be mixed up?
> 
> Kind regards, peter
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> 
> 
> 

-- 
View this message in context:
http://www.nabble.com/CVS-Dataset-Config-Usage--tf4116129.html#a11921316
Sent from the JMeter - User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org




This email (and any attachments) contains confidential information, and is intended 
only for the named recipient. Distribution or copying of this email by anyone 
other than the named recipient is prohibited. If you are not the named or 
intended recipient, please notify TEMENOS or the sender immediately and 
permanently destroy this email (and any attachments) and all copies of it. No 
member of TEMENOS Group AG or any of its associated or affiliated companies is 
liable for any errors or omissions in the content or transmission of this 
email. Any opinions contained in this email are solely those of the author and, 
unless clearly indicated otherwise in writing, are not endorsed by any member 
of TEMENOS Group AG or any of its associated and affiliated companies. 


---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org

Re: CVS Dataset Config Usage?

[Avishek Daga <av...@gmail.com>]

Each thread has its own copy of the csv file and iterates through the file
from the start



Knut Borchart wrote:
> 
> Hi,
> 
> i am experimenting with the csv dataset config but it seems like i do
> not quite understand its inner workings. What i am trying to achieve
> is the following:
> 
> - keep a file of usernames, passwords and other config for users
> around which is always 100 lines long.
> - for testing a new jmeter script i would like use different settings
> for threads and repeat (eg. 2 threads, 10 repeats)
> 
> I thought that jmeter reads the cvs file at the beginning of the
> testcase, assigns values to the different threads (=users) and thats
> it. But that does not work. With 1 thread and 10 repeats the second
> repeat of the test gets the config for user 2 (=line 2 of csv file).
> So it seems like the file is read in every iteration.
> 
> How is the csv config thought to be used? Do i have to exactly match
> the lines in the file to the number of threads i want to use? What if
> one of the threads gets an error and is stopped? Will not the
> correlation between thread <> csv config will be mixed up?
> 
> Kind regards, peter
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/CVS-Dataset-Config-Usage--tf4116129.html#a11921316
Sent from the JMeter - User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org