You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jmeter.apache.org by Philippe Mouawad <ph...@gmail.com> on 2013/10/26 21:23:40 UTC
New DataSet implementation
Hello,
These days Cloud based testing is becoming popular and having to distribute
test data accross many servers through CSV can become painful if not
impossible.
Even without Cloud, when using distributed testing you always have to
replicate your data on all servers, which is a painful manual step.
Shouldn't we introduce a new DataSet more suitable for these use cases ?
We could do it in many different ways:
- Integrate an automatic CSV replicator, this would remain simple and would
not introduce any tier, but with cloud based it would not horizontally
scale easily
- Use a JDBC DataSet, but we would need to ensure it performs fine, and
jdbc protocol is not well suited for cloud based deployment (But it could
also be an interesting feature for Continuous Integration)
- Use a NOSQL repository (Redis seems to me the best choice) , see this
high level summary which I find interesting
http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
I have implemented a new Redis (based on Java library for Redis) DataSet
which I plan to commit if no objection.
It will introduce 2 new dependencies with Apache License:
- Jedis (http://code.google.com/p/jedis/)
- commons-pool
Thoughts ?
--
Regards.
Philippe M.
@philmdot <https://twitter.com/philmdot>
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
For information, as we didn't get consensus on commiting the piece of code
to JMeter core and I didn't want to throw away the time I spent
implementing it I have contributed it to third party jmeter-plugins
project, it should be in next version of the plugins.
https://github.com/undera/jmeter-plugins/pull/22
I still think it would have been nice as is in core, creating a common
gui/api for cloud based libs being a bit complex as config properties
really differ as long as purposes.
Regards
Philippe M.
On Thursday, December 19, 2013, Philippe Mouawad wrote:
>
>
>
> On Sat, Nov 30, 2013 at 10:52 AM, sebb <sebbaz@gmail.com<javascript:_e({}, 'cvml', 'sebbaz@gmail.com');>
> > wrote:
>
>> On 27 November 2013 21:21, Philippe Mouawad <philippe.mouawad@gmail.com<javascript:_e({}, 'cvml', 'philippe.mouawad@gmail.com');>>
>> wrote:
>> > On Tue, Nov 26, 2013 at 1:05 AM, sebb <sebbaz@gmail.com<javascript:_e({}, 'cvml', 'sebbaz@gmail.com');>>
>> wrote:
>> >
>> >> On 23 November 2013 13:48, Philippe Mouawad <
>> philippe.mouawad@gmail.com <javascript:_e({}, 'cvml',
>> 'philippe.mouawad@gmail.com');>>
>> >> wrote:
>> >> > Hello sebb,
>> >> > What's your decision on this point ?
>> >>
>> >> I don't make the decisions - they are ideally arrived at by consensus;
>> >> failing that, majority.
>> >>
>> >> > Still having objection to code commit ?
>> >>
>> >> Yes, because it seems wrong to include code for a specific database
>> >> implementation.
>> >> We should be implementing generic solutions.
>> >>
>> >> So would you be ok to go for Spring-data based implementation ?
>> > It will impact number of embedded libraries.
>>
>> There are other possibilities for NoSQL as well, for example
>> http://appscale.cs.ucsb.edu/datastores.html - AppScale
>>
> Commercial no ?
>
>> http://thrift.apache.org/ - Apache Thrift
>>
>
> Didn't dig into it yet, but could you point me to a way to do it if you
> already did ?
>
>>
>> See also
>>
>> http://www.oracle.com/technetwork/database/nosqldb/overview/nosql-api-497225.html
>>
>> http://stackoverflow.com/questions/9252034/is-there-an-emerging-standard-api-for-nosql-databases
>>
>> http://stackoverflow.com/questions/3439878/are-there-any-nosql-standards-emerging
>>
>> Given that there is no standard like JDBC available (and there may
>> never be), what I think needs to happen is to decide where in JMeter
>> such databases may be useful (probably not only CSV Dataset) and then
>> define a suitable API for this.
>> Users can then implement the API in terms of whatever database they
>> wish to use (which might include a JDBC implementation).
>> This is not something to be rushed into.
>>
>
> I don't totally agree with you, I prefer sometimes that we move forward
> with "non perfect" thing and improve it in next steps.
>
> Also I am a bit annoyed that time I spent on this development is kind of
> thrown away while it is really easy to use and have tested it rather
> thoroughly.
>
>
>>
>> In any case, whatever is decided on belongs in a later version of JMeter.
>> We first need to get out the version with various fixes to keytool etc.
>>
>
> Dev is ready and could have been commited but would have agreed to wait.
>
>
> >
> >> > Thanks
> >> > Regards
> >> > Philippe
> >> >
> >> >
> >> > On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
> >> > philippe.mouawad@gmail.com> wrote:
> >> >
> >> >> We could sue Spring-Data but it would introduce a lot of
> dependencies.
> >> >>
> >> >>
> >> >> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
> >> >>
> >> >>> On 13 November 2013 21:59, Philippe Mouawad <
> >> philippe.mouawad@gmail.com>
> >> >>> wrote:
> >> >>> > Hello sebb,
> >> >>> > Shall I commit this development or do we wait for 2.10.1 to be
> >> released
> >> >>> ?
> >> >>>
> >> >>> I am still concerned that this addition is specific to the Redis
> >> server.
> >> >>> I would much prefer to see a generic solution that can use various
> >> >>> different kinds of servers.
> >> >>>
> >> >>> > When do you plan to commit your devs on keytool ?
> >> >>>
> >> >>
> >> >> If I have time I will do it , I plan the following, if you want more
> let
> >> >> me know:
> >> >> - Add keytool path property
> >> >> - Add popup to be clear about error
> >> >> - Add info about how to fix it
> >> >>
> >> >>>
> >> >>> Sorry, have not been able to spend any time on this recently.
> >> >>>
> >> >>> > Thank you
> >> >>> > Regards
> >> >>> > Philippe
> >> >>> >
> >> >>> >
> >> >>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
> >> >>> philippe.mouawad@gmail.com
> >> >>> >> wrote:
> >> >>> >
> >> >>> >> Hello Sebb,
> >> >>> >> My answers below.
> >> >>> >>
> >> >>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> >> >>> >>
> >> >>> >>> On 2 November 2013 08:28, Philippe Mouawad <
> >> >>> philippe.mouawad@gmail.com>
> >> >>> >>> wrote:
> >> >>> >>> > Hello,
> >> >>> >>> > Can I proceed with commit these up coming days ?
> >> >>> >>>
> >> >>> >>> I'm not sure that the discussion was completed.
> >> >>> >>>
> >> >>> >>> As far as I can tell, the proposal only suits some types of
> >> >>> >>> cloud-based test, and relies on 3rd party servers to hold the
> data.
> >> >>> >>>
> >> >>> >>
> >> >>> >> No in my opinion if fits many scenarios:
> >> >>> >> - Cloud based tests, this one seems to me an important one, as
> Cloud
> >> >>> based
> >> >>> >> usage are increasing
> >> >>> >> - Distributed tests, even if it is possible to do it with CSV,
> >> having
> >> >>> the
> >> >>> >> data in a remote server is much easier to manage than having to
> >> >>> >> split/distribute on servers. Even it is true it requires some
> >> skills to
> >> >>> >> manage correctly the Redis server
> >> >>> >> - Continuous integration tests where also having the data in a
> >> >>> centralised
> >> >>> >> , remote servers is easier than managing CSVs. In this case Redis
> >> >>> server
> >> >>> >> plays the same role as a JDBC repository
> >> >>> >> - Compared to a database it should perform better for high load
> >> tests
> >> >>> >> si
>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
On Sat, Nov 30, 2013 at 10:52 AM, sebb <se...@gmail.com> wrote:
> On 27 November 2013 21:21, Philippe Mouawad <ph...@gmail.com>
> wrote:
> > On Tue, Nov 26, 2013 at 1:05 AM, sebb <se...@gmail.com> wrote:
> >
> >> On 23 November 2013 13:48, Philippe Mouawad <philippe.mouawad@gmail.com
> >
> >> wrote:
> >> > Hello sebb,
> >> > What's your decision on this point ?
> >>
> >> I don't make the decisions - they are ideally arrived at by consensus;
> >> failing that, majority.
> >>
> >> > Still having objection to code commit ?
> >>
> >> Yes, because it seems wrong to include code for a specific database
> >> implementation.
> >> We should be implementing generic solutions.
> >>
> >> So would you be ok to go for Spring-data based implementation ?
> > It will impact number of embedded libraries.
>
> There are other possibilities for NoSQL as well, for example
> http://appscale.cs.ucsb.edu/datastores.html - AppScale
>
Commercial no ?
> http://thrift.apache.org/ - Apache Thrift
>
Didn't dig into it yet, but could you point me to a way to do it if you
already did ?
>
> See also
>
> http://www.oracle.com/technetwork/database/nosqldb/overview/nosql-api-497225.html
>
> http://stackoverflow.com/questions/9252034/is-there-an-emerging-standard-api-for-nosql-databases
>
> http://stackoverflow.com/questions/3439878/are-there-any-nosql-standards-emerging
>
> Given that there is no standard like JDBC available (and there may
> never be), what I think needs to happen is to decide where in JMeter
> such databases may be useful (probably not only CSV Dataset) and then
> define a suitable API for this.
> Users can then implement the API in terms of whatever database they
> wish to use (which might include a JDBC implementation).
> This is not something to be rushed into.
>
I don't totally agree with you, I prefer sometimes that we move forward
with "non perfect" thing and improve it in next steps.
Also I am a bit annoyed that time I spent on this development is kind of
thrown away while it is really easy to use and have tested it rather
thoroughly.
>
> In any case, whatever is decided on belongs in a later version of JMeter.
> We first need to get out the version with various fixes to keytool etc.
>
Dev is ready and could have been commited but would have agreed to wait.
>
> >
> >> > Thanks
> >> > Regards
> >> > Philippe
> >> >
> >> >
> >> > On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
> >> > philippe.mouawad@gmail.com> wrote:
> >> >
> >> >> We could sue Spring-Data but it would introduce a lot of
> dependencies.
> >> >>
> >> >>
> >> >> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
> >> >>
> >> >>> On 13 November 2013 21:59, Philippe Mouawad <
> >> philippe.mouawad@gmail.com>
> >> >>> wrote:
> >> >>> > Hello sebb,
> >> >>> > Shall I commit this development or do we wait for 2.10.1 to be
> >> released
> >> >>> ?
> >> >>>
> >> >>> I am still concerned that this addition is specific to the Redis
> >> server.
> >> >>> I would much prefer to see a generic solution that can use various
> >> >>> different kinds of servers.
> >> >>>
> >> >>> > When do you plan to commit your devs on keytool ?
> >> >>>
> >> >>
> >> >> If I have time I will do it , I plan the following, if you want more
> let
> >> >> me know:
> >> >> - Add keytool path property
> >> >> - Add popup to be clear about error
> >> >> - Add info about how to fix it
> >> >>
> >> >>>
> >> >>> Sorry, have not been able to spend any time on this recently.
> >> >>>
> >> >>> > Thank you
> >> >>> > Regards
> >> >>> > Philippe
> >> >>> >
> >> >>> >
> >> >>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
> >> >>> philippe.mouawad@gmail.com
> >> >>> >> wrote:
> >> >>> >
> >> >>> >> Hello Sebb,
> >> >>> >> My answers below.
> >> >>> >>
> >> >>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> >> >>> >>
> >> >>> >>> On 2 November 2013 08:28, Philippe Mouawad <
> >> >>> philippe.mouawad@gmail.com>
> >> >>> >>> wrote:
> >> >>> >>> > Hello,
> >> >>> >>> > Can I proceed with commit these up coming days ?
> >> >>> >>>
> >> >>> >>> I'm not sure that the discussion was completed.
> >> >>> >>>
> >> >>> >>> As far as I can tell, the proposal only suits some types of
> >> >>> >>> cloud-based test, and relies on 3rd party servers to hold the
> data.
> >> >>> >>>
> >> >>> >>
> >> >>> >> No in my opinion if fits many scenarios:
> >> >>> >> - Cloud based tests, this one seems to me an important one, as
> Cloud
> >> >>> based
> >> >>> >> usage are increasing
> >> >>> >> - Distributed tests, even if it is possible to do it with CSV,
> >> having
> >> >>> the
> >> >>> >> data in a remote server is much easier to manage than having to
> >> >>> >> split/distribute on servers. Even it is true it requires some
> >> skills to
> >> >>> >> manage correctly the Redis server
> >> >>> >> - Continuous integration tests where also having the data in a
> >> >>> centralised
> >> >>> >> , remote servers is easier than managing CSVs. In this case Redis
> >> >>> server
> >> >>> >> plays the same role as a JDBC repository
> >> >>> >> - Compared to a database it should perform better for high load
> >> tests
> >> >>> >> since it's an in-memory repo (although this can be done in SQL
> >> >>> databases),
> >> >>> >> see http://redis.io/topics/benchmarks
> >> >>> >>
> >> >>> >>
> >> >>> >>> I'm not yet convinced that this is how we should be extending
> >> JMeter.
> >> >>> >>>
> >> >>> >>
> >> >>> >> I don't understand this argument, it would be another string for
> >> >>> JMeter,
> >> >>> >> my implementation is only 2 classes + properties for I18N ?
> >> >>> >>
> >> >>> >> What do you propose in this case ?
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >>> > Thanks
> >> >>> >>> > Regards
> >> >>> >>> > Philippe
> >> >>> >>> >
> >> >>> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
> >> >>> >>> >
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >> On Monday, October 28, 2013, sebb wrote:
> >> >>> >>> >>
> >> >>> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
> >> >>> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
> >> >>> >>> philippe.mouawad@gmail.com>
> >> >>> >>> >>> wrote:
> >> >>> >>> >>> >> Hello,
> >> >>> >>> >>> >> These days Cloud based testing is becoming popular and
> >> having
> >> >>> to
> >> >>> >>> >>> distribute
> >> >>> >>> >>> >> test data accross many servers through CSV can become
> >> painful
> >> >>> if
> >> >>> >>> not
> >> >>> >>> >>> >> impossible.
> >> >>> >>> >>> >>
> >> >>> >>> >>> >> Even without Cloud, when using distributed testing you
> >> always
> >> >>> have
> >> >>> >>> to
> >> >>> >>> >>> >> replicate your data on all servers, which is a painful
> >> manual
> >> >>> step.
> >> >>> >>> >>> >>
> >> >>> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for
> these
> >> >>> use
> >> >>> >>> cases
> >> >>> >>> >>> ?
> >> >>> >>> >>> >>
> >> >>> >>> >>> >> We could do it in many different ways:
> >> >>> >>> >>> >> - Integrate an automatic CSV replicator, this would
> remain
> >> >>> simple
> >> >>> >>> and
> >> >>> >>> >>> would
> >> >>> >>> >>> >> not introduce any tier, but with cloud based it would not
> >> >>> >>> horizontally
> >> >>> >>> >>> >> scale easily
> >> >>> >>> >>> >
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>> > Not sure I follow the scaling argument. The file would
> only
> >> >>> have to
> >> >>> >>> be
> >> >>> >>> >>> > copied once before the test proper starts.
> >> >>> >>> >>> > From then on, the data is accessed locally.
> >> >>> >>> >>> >
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >> The scale word was not good, In my thinking the issue is more
> >> about
> >> >>> >>> >> deploying/copying/splitting the file among nodes or cloud
> >> members.
> >> >>> A
> >> >>> >>> >> centralized access makes it far easier.
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>> > However, with a database, each node will need at least one
> >> >>> >>> connection
> >> >>> >>> >>> > to the database, and every time more data is needed there
> >> will
> >> >>> be
> >> >>> >>> >>> > network traffic.
> >> >>> >>> >>> > Or the database has to be running on the server node.
> >> >>> >>> >>> >
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >> Agree, I was not saying anything different. But as I said
> this
> >> can
> >> >>> be
> >> >>> >>> >> useful for middle or low volume
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it
> >> performs
> >> >>> >>> fine, and
> >> >>> >>> >>> >> jdbc protocol is not well suited for cloud based
> deployment
> >> >>> (But it
> >> >>> >>> >>> could
> >> >>> >>> >>> >> also be an interesting feature for Continuous
> Integration)
> >> >>> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best
> >> choice)
> >> >>> , see
> >> >>> >>> >>> this
> >> >>> >>> >>> >> high level summary which I find interesting
> >> >>> >>> >>> >>
> >> >>> >>> >>>
> >> >>> >>>
> >> >>>
> >>
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
> >> >>> >>> >>> >>
> >> >>> >>> >>> >> I have implemented a new Redis (based on Java library for
> >> >>> Redis)
> >> >>> >>> >>> DataSet
> >> >>> >>> >>> >> which I plan to commit if no objection.
> >> >>> >>> >>> >>
> >> >>> >>> >>> >> It will introduce 2 new dependencies with Apache License:
> >> >>> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
> >> >>> >>> >>> >> - commons-pool
> >> >>> >>> >>> >
> >> >>> >>> >>> > There is also a dependency on Redis, but I guess that
> would
> >> not
> >> >>> be
> >> >>> >>> >>> bundled.
> >> >>> >>> >>>
> >> >>> >>> >>>
> >> >>> >>> >> no need to anything else than jedis + commons-pool
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>> I've just noticed that Redis is provided as source which
> needs
> >> to
> >> >>> be
> >> >>> >>> >>> built before use.
> >> >>> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems
> >> to me
> >> >>> >>> >>> it's going to be much harder to install Redis.
> >> >>> >>> >>> In which case additional network traffic will be needed to
> >> access
> >> >>> the
> >> >>> >>> >>> database.
> >> >>> >>> >>>
> >> >>> >>> >>> Also there is no official Windows release.
> >> >>> >>> >>>
> >> >>> >>> >>> >> Thoughts ?
> >> >>> >>> >>> >
> >> >>> >>> >>> > Is MongoDB not suitable?
> >> >>> >>> >>> > We already include a jar for it.
> >> >>> >>> >>> >
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >> Mongodb is more document oriented and has another type of
> use
> >> >>> cases.
> >> >>> >>> One
> >> >>> >>> >> interesting feature of redis is lists where you can pop a
> line
> >> it
> >> >>> will
> >> >>> >>> not
> >> >>> >>> >> be available to next calls, it is suitable for tests that
> need
> >> >>> >>> uniqueness
> >> >>> >>> >> accross all nodes.
> >> >>> >>> >>
> >> >>> >>> >>> >> --
> >> >>> >>> >>> >> Regards.
> >> >>> >>> >>> >> Philippe M.
> >> >>> >>> >>> >> @philmdot <https://twitter.com/philmdot>
> >> >>> >>> >>>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >> --
> >> >>> >>> >> Cordialement.
> >> >>> >>> >> Philippe Mouawad.
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >
> >> >>> >>> > --
> >> >>> >>> > Cordialement.
> >> >>> >>> > Philippe Mouawad.
> >> >>> >>>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >> --
> >> >>> >> Cordialement.
> >> >>> >> Philippe Mouawad.
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >
> >> >>> >
> >> >>> > --
> >> >>> > Cordialement.
> >> >>> > Philippe Mouawad.
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Cordialement.
> >> >> Philippe Mouawad.
> >> >>
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Cordialement.
> >> > Philippe Mouawad.
> >>
> >
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by sebb <se...@gmail.com>.
On 27 November 2013 21:21, Philippe Mouawad <ph...@gmail.com> wrote:
> On Tue, Nov 26, 2013 at 1:05 AM, sebb <se...@gmail.com> wrote:
>
>> On 23 November 2013 13:48, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>> > Hello sebb,
>> > What's your decision on this point ?
>>
>> I don't make the decisions - they are ideally arrived at by consensus;
>> failing that, majority.
>>
>> > Still having objection to code commit ?
>>
>> Yes, because it seems wrong to include code for a specific database
>> implementation.
>> We should be implementing generic solutions.
>>
>> So would you be ok to go for Spring-data based implementation ?
> It will impact number of embedded libraries.
There are other possibilities for NoSQL as well, for example
http://appscale.cs.ucsb.edu/datastores.html - AppScale
http://thrift.apache.org/ - Apache Thrift
See also
http://www.oracle.com/technetwork/database/nosqldb/overview/nosql-api-497225.html
http://stackoverflow.com/questions/9252034/is-there-an-emerging-standard-api-for-nosql-databases
http://stackoverflow.com/questions/3439878/are-there-any-nosql-standards-emerging
Given that there is no standard like JDBC available (and there may
never be), what I think needs to happen is to decide where in JMeter
such databases may be useful (probably not only CSV Dataset) and then
define a suitable API for this.
Users can then implement the API in terms of whatever database they
wish to use (which might include a JDBC implementation).
This is not something to be rushed into.
In any case, whatever is decided on belongs in a later version of JMeter.
We first need to get out the version with various fixes to keytool etc.
>
>> > Thanks
>> > Regards
>> > Philippe
>> >
>> >
>> > On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
>> > philippe.mouawad@gmail.com> wrote:
>> >
>> >> We could sue Spring-Data but it would introduce a lot of dependencies.
>> >>
>> >>
>> >> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
>> >>
>> >>> On 13 November 2013 21:59, Philippe Mouawad <
>> philippe.mouawad@gmail.com>
>> >>> wrote:
>> >>> > Hello sebb,
>> >>> > Shall I commit this development or do we wait for 2.10.1 to be
>> released
>> >>> ?
>> >>>
>> >>> I am still concerned that this addition is specific to the Redis
>> server.
>> >>> I would much prefer to see a generic solution that can use various
>> >>> different kinds of servers.
>> >>>
>> >>> > When do you plan to commit your devs on keytool ?
>> >>>
>> >>
>> >> If I have time I will do it , I plan the following, if you want more let
>> >> me know:
>> >> - Add keytool path property
>> >> - Add popup to be clear about error
>> >> - Add info about how to fix it
>> >>
>> >>>
>> >>> Sorry, have not been able to spend any time on this recently.
>> >>>
>> >>> > Thank you
>> >>> > Regards
>> >>> > Philippe
>> >>> >
>> >>> >
>> >>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
>> >>> philippe.mouawad@gmail.com
>> >>> >> wrote:
>> >>> >
>> >>> >> Hello Sebb,
>> >>> >> My answers below.
>> >>> >>
>> >>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
>> >>> >>
>> >>> >>> On 2 November 2013 08:28, Philippe Mouawad <
>> >>> philippe.mouawad@gmail.com>
>> >>> >>> wrote:
>> >>> >>> > Hello,
>> >>> >>> > Can I proceed with commit these up coming days ?
>> >>> >>>
>> >>> >>> I'm not sure that the discussion was completed.
>> >>> >>>
>> >>> >>> As far as I can tell, the proposal only suits some types of
>> >>> >>> cloud-based test, and relies on 3rd party servers to hold the data.
>> >>> >>>
>> >>> >>
>> >>> >> No in my opinion if fits many scenarios:
>> >>> >> - Cloud based tests, this one seems to me an important one, as Cloud
>> >>> based
>> >>> >> usage are increasing
>> >>> >> - Distributed tests, even if it is possible to do it with CSV,
>> having
>> >>> the
>> >>> >> data in a remote server is much easier to manage than having to
>> >>> >> split/distribute on servers. Even it is true it requires some
>> skills to
>> >>> >> manage correctly the Redis server
>> >>> >> - Continuous integration tests where also having the data in a
>> >>> centralised
>> >>> >> , remote servers is easier than managing CSVs. In this case Redis
>> >>> server
>> >>> >> plays the same role as a JDBC repository
>> >>> >> - Compared to a database it should perform better for high load
>> tests
>> >>> >> since it's an in-memory repo (although this can be done in SQL
>> >>> databases),
>> >>> >> see http://redis.io/topics/benchmarks
>> >>> >>
>> >>> >>
>> >>> >>> I'm not yet convinced that this is how we should be extending
>> JMeter.
>> >>> >>>
>> >>> >>
>> >>> >> I don't understand this argument, it would be another string for
>> >>> JMeter,
>> >>> >> my implementation is only 2 classes + properties for I18N ?
>> >>> >>
>> >>> >> What do you propose in this case ?
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>> > Thanks
>> >>> >>> > Regards
>> >>> >>> > Philippe
>> >>> >>> >
>> >>> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> On Monday, October 28, 2013, sebb wrote:
>> >>> >>> >>
>> >>> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>> >>> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
>> >>> >>> philippe.mouawad@gmail.com>
>> >>> >>> >>> wrote:
>> >>> >>> >>> >> Hello,
>> >>> >>> >>> >> These days Cloud based testing is becoming popular and
>> having
>> >>> to
>> >>> >>> >>> distribute
>> >>> >>> >>> >> test data accross many servers through CSV can become
>> painful
>> >>> if
>> >>> >>> not
>> >>> >>> >>> >> impossible.
>> >>> >>> >>> >>
>> >>> >>> >>> >> Even without Cloud, when using distributed testing you
>> always
>> >>> have
>> >>> >>> to
>> >>> >>> >>> >> replicate your data on all servers, which is a painful
>> manual
>> >>> step.
>> >>> >>> >>> >>
>> >>> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for these
>> >>> use
>> >>> >>> cases
>> >>> >>> >>> ?
>> >>> >>> >>> >>
>> >>> >>> >>> >> We could do it in many different ways:
>> >>> >>> >>> >> - Integrate an automatic CSV replicator, this would remain
>> >>> simple
>> >>> >>> and
>> >>> >>> >>> would
>> >>> >>> >>> >> not introduce any tier, but with cloud based it would not
>> >>> >>> horizontally
>> >>> >>> >>> >> scale easily
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> > Not sure I follow the scaling argument. The file would only
>> >>> have to
>> >>> >>> be
>> >>> >>> >>> > copied once before the test proper starts.
>> >>> >>> >>> > From then on, the data is accessed locally.
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> The scale word was not good, In my thinking the issue is more
>> about
>> >>> >>> >> deploying/copying/splitting the file among nodes or cloud
>> members.
>> >>> A
>> >>> >>> >> centralized access makes it far easier.
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> > However, with a database, each node will need at least one
>> >>> >>> connection
>> >>> >>> >>> > to the database, and every time more data is needed there
>> will
>> >>> be
>> >>> >>> >>> > network traffic.
>> >>> >>> >>> > Or the database has to be running on the server node.
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> Agree, I was not saying anything different. But as I said this
>> can
>> >>> be
>> >>> >>> >> useful for middle or low volume
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it
>> performs
>> >>> >>> fine, and
>> >>> >>> >>> >> jdbc protocol is not well suited for cloud based deployment
>> >>> (But it
>> >>> >>> >>> could
>> >>> >>> >>> >> also be an interesting feature for Continuous Integration)
>> >>> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best
>> choice)
>> >>> , see
>> >>> >>> >>> this
>> >>> >>> >>> >> high level summary which I find interesting
>> >>> >>> >>> >>
>> >>> >>> >>>
>> >>> >>>
>> >>>
>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>> >>> >>> >>> >>
>> >>> >>> >>> >> I have implemented a new Redis (based on Java library for
>> >>> Redis)
>> >>> >>> >>> DataSet
>> >>> >>> >>> >> which I plan to commit if no objection.
>> >>> >>> >>> >>
>> >>> >>> >>> >> It will introduce 2 new dependencies with Apache License:
>> >>> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
>> >>> >>> >>> >> - commons-pool
>> >>> >>> >>> >
>> >>> >>> >>> > There is also a dependency on Redis, but I guess that would
>> not
>> >>> be
>> >>> >>> >>> bundled.
>> >>> >>> >>>
>> >>> >>> >>>
>> >>> >>> >> no need to anything else than jedis + commons-pool
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> I've just noticed that Redis is provided as source which needs
>> to
>> >>> be
>> >>> >>> >>> built before use.
>> >>> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems
>> to me
>> >>> >>> >>> it's going to be much harder to install Redis.
>> >>> >>> >>> In which case additional network traffic will be needed to
>> access
>> >>> the
>> >>> >>> >>> database.
>> >>> >>> >>>
>> >>> >>> >>> Also there is no official Windows release.
>> >>> >>> >>>
>> >>> >>> >>> >> Thoughts ?
>> >>> >>> >>> >
>> >>> >>> >>> > Is MongoDB not suitable?
>> >>> >>> >>> > We already include a jar for it.
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> Mongodb is more document oriented and has another type of use
>> >>> cases.
>> >>> >>> One
>> >>> >>> >> interesting feature of redis is lists where you can pop a line
>> it
>> >>> will
>> >>> >>> not
>> >>> >>> >> be available to next calls, it is suitable for tests that need
>> >>> >>> uniqueness
>> >>> >>> >> accross all nodes.
>> >>> >>> >>
>> >>> >>> >>> >> --
>> >>> >>> >>> >> Regards.
>> >>> >>> >>> >> Philippe M.
>> >>> >>> >>> >> @philmdot <https://twitter.com/philmdot>
>> >>> >>> >>>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> --
>> >>> >>> >> Cordialement.
>> >>> >>> >> Philippe Mouawad.
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >
>> >>> >>> > --
>> >>> >>> > Cordialement.
>> >>> >>> > Philippe Mouawad.
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Cordialement.
>> >>> >> Philippe Mouawad.
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Cordialement.
>> >>> > Philippe Mouawad.
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Cordialement.
>> >> Philippe Mouawad.
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
Any feedback?
Thanks
On Wednesday, November 27, 2013, Philippe Mouawad wrote:
>
>
>
> On Tue, Nov 26, 2013 at 1:05 AM, sebb <sebbaz@gmail.com<javascript:_e({}, 'cvml', 'sebbaz@gmail.com');>
> > wrote:
>
>> On 23 November 2013 13:48, Philippe Mouawad <philippe.mouawad@gmail.com<javascript:_e({}, 'cvml', 'philippe.mouawad@gmail.com');>>
>> wrote:
>> > Hello sebb,
>> > What's your decision on this point ?
>>
>> I don't make the decisions - they are ideally arrived at by consensus;
>> failing that, majority.
>>
>> > Still having objection to code commit ?
>>
>> Yes, because it seems wrong to include code for a specific database
>> implementation.
>> We should be implementing generic solutions.
>>
>> So would you be ok to go for Spring-data based implementation ?
> It will impact number of embedded libraries.
>
>
> > Thanks
> > Regards
> > Philippe
> >
> >
> > On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
> > philippe.mouawad@gmail.com> wrote:
> >
> >> We could sue Spring-Data but it would introduce a lot of dependencies.
> >>
> >>
> >> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
> >>
> >>> On 13 November 2013 21:59, Philippe Mouawad <
> philippe.mouawad@gmail.com>
> >>> wrote:
> >>> > Hello sebb,
> >>> > Shall I commit this development or do we wait for 2.10.1 to be
> released
> >>> ?
> >>>
> >>> I am still concerned that this addition is specific to the Redis
> server.
> >>> I would much prefer to see a generic solution that can use various
> >>> different kinds of servers.
> >>>
> >>> > When do you plan to commit your devs on keytool ?
> >>>
> >>
> >> If I have time I will do it , I plan the following, if you want more let
> >> me know:
> >> - Add keytool path property
> >> - Add popup to be clear about error
> >> - Add info about how to fix it
> >>
> >>>
> >>> Sorry, have not been able to spend any time on this recently.
> >>>
> >>> > Thank you
> >>> > Regards
> >>> > Philippe
> >>> >
> >>> >
> >>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
> >>> philippe.mouawad@gmail.com
> >>> >> wrote:
> >>> >
> >>> >> Hello Sebb,
> >>> >> My answers below.
> >>> >>
> >>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> >>> >>
> >>> >>> On 2 November 2013 08:28, Philippe Mouawad <
> >>> philippe.mouawad@gmail.com>
> >>> >>> wrote:
> >>> >>> > Hello,
> >>> >>> > Can I proceed with commit these up coming days ?
> >>> >>>
> >>> >>> I'm not sure that the discussion was completed.
> >>> >>>
> >>> >>> As far as I can tell, the proposal only suits some types of
> >>> >>> cloud-based test, and relies on 3rd party servers to hold the data.
> >>> >>>
> >>> >>
> >>> >> No in my opinion if fits many scenarios:
> >>> >> - Cloud based tests, this one seems to me an important one, as Cloud
> >>> based
> >>> >> usage are increasing
> >>> >> - Distributed tests, even if it is possible to do it with CSV,
> having
> >>> the
> >>> >> data in a remote server is much easier to manage than having to
> >>> >> split/distribute on servers. Even it is true it requires some
> skills to
> >>> >> manage correctly the Redis server
> >>> >> - Continuous integration tests where also having the data in a
> >>> centralised
> >>> >> , remote servers is easier than managing CSVs. In this case Redis
> >>> server
> >>> >> plays the same role as a JDBC repository
> >>> >> - Compared to a database it should perform better for high load
> tests
> >>> >> since it's an in-memory repo (although this can be done in SQL
> >>> databases),
> >>> >> see http://redis.io/topics/benchmarks
> >>> >>
> >>> >>
> >>> >>> I'm not yet convinced that this is how we should be extending
> JMeter.
> >>> >>>
> >>> >>
> >>> >> I don't understand this argument, it would be another string for
> >>> JMeter,
> >>> >> my implementation is only 2 classes + properties for I18N ?
> >>> >>
> >>> >> What do you propose in this case ?
> >>> >>
> >>
>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
On Tue, Nov 26, 2013 at 1:05 AM, sebb <se...@gmail.com> wrote:
> On 23 November 2013 13:48, Philippe Mouawad <ph...@gmail.com>
> wrote:
> > Hello sebb,
> > What's your decision on this point ?
>
> I don't make the decisions - they are ideally arrived at by consensus;
> failing that, majority.
>
> > Still having objection to code commit ?
>
> Yes, because it seems wrong to include code for a specific database
> implementation.
> We should be implementing generic solutions.
>
> So would you be ok to go for Spring-data based implementation ?
It will impact number of embedded libraries.
> > Thanks
> > Regards
> > Philippe
> >
> >
> > On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
> > philippe.mouawad@gmail.com> wrote:
> >
> >> We could sue Spring-Data but it would introduce a lot of dependencies.
> >>
> >>
> >> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
> >>
> >>> On 13 November 2013 21:59, Philippe Mouawad <
> philippe.mouawad@gmail.com>
> >>> wrote:
> >>> > Hello sebb,
> >>> > Shall I commit this development or do we wait for 2.10.1 to be
> released
> >>> ?
> >>>
> >>> I am still concerned that this addition is specific to the Redis
> server.
> >>> I would much prefer to see a generic solution that can use various
> >>> different kinds of servers.
> >>>
> >>> > When do you plan to commit your devs on keytool ?
> >>>
> >>
> >> If I have time I will do it , I plan the following, if you want more let
> >> me know:
> >> - Add keytool path property
> >> - Add popup to be clear about error
> >> - Add info about how to fix it
> >>
> >>>
> >>> Sorry, have not been able to spend any time on this recently.
> >>>
> >>> > Thank you
> >>> > Regards
> >>> > Philippe
> >>> >
> >>> >
> >>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
> >>> philippe.mouawad@gmail.com
> >>> >> wrote:
> >>> >
> >>> >> Hello Sebb,
> >>> >> My answers below.
> >>> >>
> >>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> >>> >>
> >>> >>> On 2 November 2013 08:28, Philippe Mouawad <
> >>> philippe.mouawad@gmail.com>
> >>> >>> wrote:
> >>> >>> > Hello,
> >>> >>> > Can I proceed with commit these up coming days ?
> >>> >>>
> >>> >>> I'm not sure that the discussion was completed.
> >>> >>>
> >>> >>> As far as I can tell, the proposal only suits some types of
> >>> >>> cloud-based test, and relies on 3rd party servers to hold the data.
> >>> >>>
> >>> >>
> >>> >> No in my opinion if fits many scenarios:
> >>> >> - Cloud based tests, this one seems to me an important one, as Cloud
> >>> based
> >>> >> usage are increasing
> >>> >> - Distributed tests, even if it is possible to do it with CSV,
> having
> >>> the
> >>> >> data in a remote server is much easier to manage than having to
> >>> >> split/distribute on servers. Even it is true it requires some
> skills to
> >>> >> manage correctly the Redis server
> >>> >> - Continuous integration tests where also having the data in a
> >>> centralised
> >>> >> , remote servers is easier than managing CSVs. In this case Redis
> >>> server
> >>> >> plays the same role as a JDBC repository
> >>> >> - Compared to a database it should perform better for high load
> tests
> >>> >> since it's an in-memory repo (although this can be done in SQL
> >>> databases),
> >>> >> see http://redis.io/topics/benchmarks
> >>> >>
> >>> >>
> >>> >>> I'm not yet convinced that this is how we should be extending
> JMeter.
> >>> >>>
> >>> >>
> >>> >> I don't understand this argument, it would be another string for
> >>> JMeter,
> >>> >> my implementation is only 2 classes + properties for I18N ?
> >>> >>
> >>> >> What do you propose in this case ?
> >>> >>
> >>> >>
> >>> >>
> >>> >>> > Thanks
> >>> >>> > Regards
> >>> >>> > Philippe
> >>> >>> >
> >>> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
> >>> >>> >
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> On Monday, October 28, 2013, sebb wrote:
> >>> >>> >>
> >>> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
> >>> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
> >>> >>> philippe.mouawad@gmail.com>
> >>> >>> >>> wrote:
> >>> >>> >>> >> Hello,
> >>> >>> >>> >> These days Cloud based testing is becoming popular and
> having
> >>> to
> >>> >>> >>> distribute
> >>> >>> >>> >> test data accross many servers through CSV can become
> painful
> >>> if
> >>> >>> not
> >>> >>> >>> >> impossible.
> >>> >>> >>> >>
> >>> >>> >>> >> Even without Cloud, when using distributed testing you
> always
> >>> have
> >>> >>> to
> >>> >>> >>> >> replicate your data on all servers, which is a painful
> manual
> >>> step.
> >>> >>> >>> >>
> >>> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for these
> >>> use
> >>> >>> cases
> >>> >>> >>> ?
> >>> >>> >>> >>
> >>> >>> >>> >> We could do it in many different ways:
> >>> >>> >>> >> - Integrate an automatic CSV replicator, this would remain
> >>> simple
> >>> >>> and
> >>> >>> >>> would
> >>> >>> >>> >> not introduce any tier, but with cloud based it would not
> >>> >>> horizontally
> >>> >>> >>> >> scale easily
> >>> >>> >>> >
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>> > Not sure I follow the scaling argument. The file would only
> >>> have to
> >>> >>> be
> >>> >>> >>> > copied once before the test proper starts.
> >>> >>> >>> > From then on, the data is accessed locally.
> >>> >>> >>> >
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> The scale word was not good, In my thinking the issue is more
> about
> >>> >>> >> deploying/copying/splitting the file among nodes or cloud
> members.
> >>> A
> >>> >>> >> centralized access makes it far easier.
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>> > However, with a database, each node will need at least one
> >>> >>> connection
> >>> >>> >>> > to the database, and every time more data is needed there
> will
> >>> be
> >>> >>> >>> > network traffic.
> >>> >>> >>> > Or the database has to be running on the server node.
> >>> >>> >>> >
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> Agree, I was not saying anything different. But as I said this
> can
> >>> be
> >>> >>> >> useful for middle or low volume
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it
> performs
> >>> >>> fine, and
> >>> >>> >>> >> jdbc protocol is not well suited for cloud based deployment
> >>> (But it
> >>> >>> >>> could
> >>> >>> >>> >> also be an interesting feature for Continuous Integration)
> >>> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best
> choice)
> >>> , see
> >>> >>> >>> this
> >>> >>> >>> >> high level summary which I find interesting
> >>> >>> >>> >>
> >>> >>> >>>
> >>> >>>
> >>>
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
> >>> >>> >>> >>
> >>> >>> >>> >> I have implemented a new Redis (based on Java library for
> >>> Redis)
> >>> >>> >>> DataSet
> >>> >>> >>> >> which I plan to commit if no objection.
> >>> >>> >>> >>
> >>> >>> >>> >> It will introduce 2 new dependencies with Apache License:
> >>> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
> >>> >>> >>> >> - commons-pool
> >>> >>> >>> >
> >>> >>> >>> > There is also a dependency on Redis, but I guess that would
> not
> >>> be
> >>> >>> >>> bundled.
> >>> >>> >>>
> >>> >>> >>>
> >>> >>> >> no need to anything else than jedis + commons-pool
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>> I've just noticed that Redis is provided as source which needs
> to
> >>> be
> >>> >>> >>> built before use.
> >>> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems
> to me
> >>> >>> >>> it's going to be much harder to install Redis.
> >>> >>> >>> In which case additional network traffic will be needed to
> access
> >>> the
> >>> >>> >>> database.
> >>> >>> >>>
> >>> >>> >>> Also there is no official Windows release.
> >>> >>> >>>
> >>> >>> >>> >> Thoughts ?
> >>> >>> >>> >
> >>> >>> >>> > Is MongoDB not suitable?
> >>> >>> >>> > We already include a jar for it.
> >>> >>> >>> >
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> Mongodb is more document oriented and has another type of use
> >>> cases.
> >>> >>> One
> >>> >>> >> interesting feature of redis is lists where you can pop a line
> it
> >>> will
> >>> >>> not
> >>> >>> >> be available to next calls, it is suitable for tests that need
> >>> >>> uniqueness
> >>> >>> >> accross all nodes.
> >>> >>> >>
> >>> >>> >>> >> --
> >>> >>> >>> >> Regards.
> >>> >>> >>> >> Philippe M.
> >>> >>> >>> >> @philmdot <https://twitter.com/philmdot>
> >>> >>> >>>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> --
> >>> >>> >> Cordialement.
> >>> >>> >> Philippe Mouawad.
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >
> >>> >>> > --
> >>> >>> > Cordialement.
> >>> >>> > Philippe Mouawad.
> >>> >>>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Cordialement.
> >>> >> Philippe Mouawad.
> >>> >>
> >>> >>
> >>> >>
> >>> >
> >>> >
> >>> > --
> >>> > Cordialement.
> >>> > Philippe Mouawad.
> >>>
> >>
> >>
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >>
> >>
> >>
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by sebb <se...@gmail.com>.
On 23 November 2013 13:48, Philippe Mouawad <ph...@gmail.com> wrote:
> Hello sebb,
> What's your decision on this point ?
I don't make the decisions - they are ideally arrived at by consensus;
failing that, majority.
> Still having objection to code commit ?
Yes, because it seems wrong to include code for a specific database
implementation.
We should be implementing generic solutions.
> Thanks
> Regards
> Philippe
>
>
> On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
> philippe.mouawad@gmail.com> wrote:
>
>> We could sue Spring-Data but it would introduce a lot of dependencies.
>>
>>
>> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
>>
>>> On 13 November 2013 21:59, Philippe Mouawad <ph...@gmail.com>
>>> wrote:
>>> > Hello sebb,
>>> > Shall I commit this development or do we wait for 2.10.1 to be released
>>> ?
>>>
>>> I am still concerned that this addition is specific to the Redis server.
>>> I would much prefer to see a generic solution that can use various
>>> different kinds of servers.
>>>
>>> > When do you plan to commit your devs on keytool ?
>>>
>>
>> If I have time I will do it , I plan the following, if you want more let
>> me know:
>> - Add keytool path property
>> - Add popup to be clear about error
>> - Add info about how to fix it
>>
>>>
>>> Sorry, have not been able to spend any time on this recently.
>>>
>>> > Thank you
>>> > Regards
>>> > Philippe
>>> >
>>> >
>>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
>>> philippe.mouawad@gmail.com
>>> >> wrote:
>>> >
>>> >> Hello Sebb,
>>> >> My answers below.
>>> >>
>>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
>>> >>
>>> >>> On 2 November 2013 08:28, Philippe Mouawad <
>>> philippe.mouawad@gmail.com>
>>> >>> wrote:
>>> >>> > Hello,
>>> >>> > Can I proceed with commit these up coming days ?
>>> >>>
>>> >>> I'm not sure that the discussion was completed.
>>> >>>
>>> >>> As far as I can tell, the proposal only suits some types of
>>> >>> cloud-based test, and relies on 3rd party servers to hold the data.
>>> >>>
>>> >>
>>> >> No in my opinion if fits many scenarios:
>>> >> - Cloud based tests, this one seems to me an important one, as Cloud
>>> based
>>> >> usage are increasing
>>> >> - Distributed tests, even if it is possible to do it with CSV, having
>>> the
>>> >> data in a remote server is much easier to manage than having to
>>> >> split/distribute on servers. Even it is true it requires some skills to
>>> >> manage correctly the Redis server
>>> >> - Continuous integration tests where also having the data in a
>>> centralised
>>> >> , remote servers is easier than managing CSVs. In this case Redis
>>> server
>>> >> plays the same role as a JDBC repository
>>> >> - Compared to a database it should perform better for high load tests
>>> >> since it's an in-memory repo (although this can be done in SQL
>>> databases),
>>> >> see http://redis.io/topics/benchmarks
>>> >>
>>> >>
>>> >>> I'm not yet convinced that this is how we should be extending JMeter.
>>> >>>
>>> >>
>>> >> I don't understand this argument, it would be another string for
>>> JMeter,
>>> >> my implementation is only 2 classes + properties for I18N ?
>>> >>
>>> >> What do you propose in this case ?
>>> >>
>>> >>
>>> >>
>>> >>> > Thanks
>>> >>> > Regards
>>> >>> > Philippe
>>> >>> >
>>> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
>>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >> On Monday, October 28, 2013, sebb wrote:
>>> >>> >>
>>> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>>> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
>>> >>> philippe.mouawad@gmail.com>
>>> >>> >>> wrote:
>>> >>> >>> >> Hello,
>>> >>> >>> >> These days Cloud based testing is becoming popular and having
>>> to
>>> >>> >>> distribute
>>> >>> >>> >> test data accross many servers through CSV can become painful
>>> if
>>> >>> not
>>> >>> >>> >> impossible.
>>> >>> >>> >>
>>> >>> >>> >> Even without Cloud, when using distributed testing you always
>>> have
>>> >>> to
>>> >>> >>> >> replicate your data on all servers, which is a painful manual
>>> step.
>>> >>> >>> >>
>>> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for these
>>> use
>>> >>> cases
>>> >>> >>> ?
>>> >>> >>> >>
>>> >>> >>> >> We could do it in many different ways:
>>> >>> >>> >> - Integrate an automatic CSV replicator, this would remain
>>> simple
>>> >>> and
>>> >>> >>> would
>>> >>> >>> >> not introduce any tier, but with cloud based it would not
>>> >>> horizontally
>>> >>> >>> >> scale easily
>>> >>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >>> > Not sure I follow the scaling argument. The file would only
>>> have to
>>> >>> be
>>> >>> >>> > copied once before the test proper starts.
>>> >>> >>> > From then on, the data is accessed locally.
>>> >>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >> The scale word was not good, In my thinking the issue is more about
>>> >>> >> deploying/copying/splitting the file among nodes or cloud members.
>>> A
>>> >>> >> centralized access makes it far easier.
>>> >>> >>
>>> >>> >>
>>> >>> >>> > However, with a database, each node will need at least one
>>> >>> connection
>>> >>> >>> > to the database, and every time more data is needed there will
>>> be
>>> >>> >>> > network traffic.
>>> >>> >>> > Or the database has to be running on the server node.
>>> >>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >> Agree, I was not saying anything different. But as I said this can
>>> be
>>> >>> >> useful for middle or low volume
>>> >>> >>
>>> >>> >>
>>> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs
>>> >>> fine, and
>>> >>> >>> >> jdbc protocol is not well suited for cloud based deployment
>>> (But it
>>> >>> >>> could
>>> >>> >>> >> also be an interesting feature for Continuous Integration)
>>> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best choice)
>>> , see
>>> >>> >>> this
>>> >>> >>> >> high level summary which I find interesting
>>> >>> >>> >>
>>> >>> >>>
>>> >>>
>>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>>> >>> >>> >>
>>> >>> >>> >> I have implemented a new Redis (based on Java library for
>>> Redis)
>>> >>> >>> DataSet
>>> >>> >>> >> which I plan to commit if no objection.
>>> >>> >>> >>
>>> >>> >>> >> It will introduce 2 new dependencies with Apache License:
>>> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
>>> >>> >>> >> - commons-pool
>>> >>> >>> >
>>> >>> >>> > There is also a dependency on Redis, but I guess that would not
>>> be
>>> >>> >>> bundled.
>>> >>> >>>
>>> >>> >>>
>>> >>> >> no need to anything else than jedis + commons-pool
>>> >>> >>
>>> >>> >>
>>> >>> >>> I've just noticed that Redis is provided as source which needs to
>>> be
>>> >>> >>> built before use.
>>> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>>> >>> >>> it's going to be much harder to install Redis.
>>> >>> >>> In which case additional network traffic will be needed to access
>>> the
>>> >>> >>> database.
>>> >>> >>>
>>> >>> >>> Also there is no official Windows release.
>>> >>> >>>
>>> >>> >>> >> Thoughts ?
>>> >>> >>> >
>>> >>> >>> > Is MongoDB not suitable?
>>> >>> >>> > We already include a jar for it.
>>> >>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >> Mongodb is more document oriented and has another type of use
>>> cases.
>>> >>> One
>>> >>> >> interesting feature of redis is lists where you can pop a line it
>>> will
>>> >>> not
>>> >>> >> be available to next calls, it is suitable for tests that need
>>> >>> uniqueness
>>> >>> >> accross all nodes.
>>> >>> >>
>>> >>> >>> >> --
>>> >>> >>> >> Regards.
>>> >>> >>> >> Philippe M.
>>> >>> >>> >> @philmdot <https://twitter.com/philmdot>
>>> >>> >>>
>>> >>> >>
>>> >>> >>
>>> >>> >> --
>>> >>> >> Cordialement.
>>> >>> >> Philippe Mouawad.
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >
>>> >>> > --
>>> >>> > Cordialement.
>>> >>> > Philippe Mouawad.
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Cordialement.
>>> >> Philippe Mouawad.
>>> >>
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> > Cordialement.
>>> > Philippe Mouawad.
>>>
>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello sebb,
What's your decision on this point ?
Still having objection to code commit ?
Thanks
Regards
Philippe
On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:
> We could sue Spring-Data but it would introduce a lot of dependencies.
>
>
> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
>
>> On 13 November 2013 21:59, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>> > Hello sebb,
>> > Shall I commit this development or do we wait for 2.10.1 to be released
>> ?
>>
>> I am still concerned that this addition is specific to the Redis server.
>> I would much prefer to see a generic solution that can use various
>> different kinds of servers.
>>
>> > When do you plan to commit your devs on keytool ?
>>
>
> If I have time I will do it , I plan the following, if you want more let
> me know:
> - Add keytool path property
> - Add popup to be clear about error
> - Add info about how to fix it
>
>>
>> Sorry, have not been able to spend any time on this recently.
>>
>> > Thank you
>> > Regards
>> > Philippe
>> >
>> >
>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
>> philippe.mouawad@gmail.com
>> >> wrote:
>> >
>> >> Hello Sebb,
>> >> My answers below.
>> >>
>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
>> >>
>> >>> On 2 November 2013 08:28, Philippe Mouawad <
>> philippe.mouawad@gmail.com>
>> >>> wrote:
>> >>> > Hello,
>> >>> > Can I proceed with commit these up coming days ?
>> >>>
>> >>> I'm not sure that the discussion was completed.
>> >>>
>> >>> As far as I can tell, the proposal only suits some types of
>> >>> cloud-based test, and relies on 3rd party servers to hold the data.
>> >>>
>> >>
>> >> No in my opinion if fits many scenarios:
>> >> - Cloud based tests, this one seems to me an important one, as Cloud
>> based
>> >> usage are increasing
>> >> - Distributed tests, even if it is possible to do it with CSV, having
>> the
>> >> data in a remote server is much easier to manage than having to
>> >> split/distribute on servers. Even it is true it requires some skills to
>> >> manage correctly the Redis server
>> >> - Continuous integration tests where also having the data in a
>> centralised
>> >> , remote servers is easier than managing CSVs. In this case Redis
>> server
>> >> plays the same role as a JDBC repository
>> >> - Compared to a database it should perform better for high load tests
>> >> since it's an in-memory repo (although this can be done in SQL
>> databases),
>> >> see http://redis.io/topics/benchmarks
>> >>
>> >>
>> >>> I'm not yet convinced that this is how we should be extending JMeter.
>> >>>
>> >>
>> >> I don't understand this argument, it would be another string for
>> JMeter,
>> >> my implementation is only 2 classes + properties for I18N ?
>> >>
>> >> What do you propose in this case ?
>> >>
>> >>
>> >>
>> >>> > Thanks
>> >>> > Regards
>> >>> > Philippe
>> >>> >
>> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
>> >>> >
>> >>> >>
>> >>> >>
>> >>> >> On Monday, October 28, 2013, sebb wrote:
>> >>> >>
>> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
>> >>> philippe.mouawad@gmail.com>
>> >>> >>> wrote:
>> >>> >>> >> Hello,
>> >>> >>> >> These days Cloud based testing is becoming popular and having
>> to
>> >>> >>> distribute
>> >>> >>> >> test data accross many servers through CSV can become painful
>> if
>> >>> not
>> >>> >>> >> impossible.
>> >>> >>> >>
>> >>> >>> >> Even without Cloud, when using distributed testing you always
>> have
>> >>> to
>> >>> >>> >> replicate your data on all servers, which is a painful manual
>> step.
>> >>> >>> >>
>> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for these
>> use
>> >>> cases
>> >>> >>> ?
>> >>> >>> >>
>> >>> >>> >> We could do it in many different ways:
>> >>> >>> >> - Integrate an automatic CSV replicator, this would remain
>> simple
>> >>> and
>> >>> >>> would
>> >>> >>> >> not introduce any tier, but with cloud based it would not
>> >>> horizontally
>> >>> >>> >> scale easily
>> >>> >>> >
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>> > Not sure I follow the scaling argument. The file would only
>> have to
>> >>> be
>> >>> >>> > copied once before the test proper starts.
>> >>> >>> > From then on, the data is accessed locally.
>> >>> >>> >
>> >>> >>
>> >>> >>
>> >>> >> The scale word was not good, In my thinking the issue is more about
>> >>> >> deploying/copying/splitting the file among nodes or cloud members.
>> A
>> >>> >> centralized access makes it far easier.
>> >>> >>
>> >>> >>
>> >>> >>> > However, with a database, each node will need at least one
>> >>> connection
>> >>> >>> > to the database, and every time more data is needed there will
>> be
>> >>> >>> > network traffic.
>> >>> >>> > Or the database has to be running on the server node.
>> >>> >>> >
>> >>> >>
>> >>> >>
>> >>> >> Agree, I was not saying anything different. But as I said this can
>> be
>> >>> >> useful for middle or low volume
>> >>> >>
>> >>> >>
>> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs
>> >>> fine, and
>> >>> >>> >> jdbc protocol is not well suited for cloud based deployment
>> (But it
>> >>> >>> could
>> >>> >>> >> also be an interesting feature for Continuous Integration)
>> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best choice)
>> , see
>> >>> >>> this
>> >>> >>> >> high level summary which I find interesting
>> >>> >>> >>
>> >>> >>>
>> >>>
>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>> >>> >>> >>
>> >>> >>> >> I have implemented a new Redis (based on Java library for
>> Redis)
>> >>> >>> DataSet
>> >>> >>> >> which I plan to commit if no objection.
>> >>> >>> >>
>> >>> >>> >> It will introduce 2 new dependencies with Apache License:
>> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
>> >>> >>> >> - commons-pool
>> >>> >>> >
>> >>> >>> > There is also a dependency on Redis, but I guess that would not
>> be
>> >>> >>> bundled.
>> >>> >>>
>> >>> >>>
>> >>> >> no need to anything else than jedis + commons-pool
>> >>> >>
>> >>> >>
>> >>> >>> I've just noticed that Redis is provided as source which needs to
>> be
>> >>> >>> built before use.
>> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>> >>> >>> it's going to be much harder to install Redis.
>> >>> >>> In which case additional network traffic will be needed to access
>> the
>> >>> >>> database.
>> >>> >>>
>> >>> >>> Also there is no official Windows release.
>> >>> >>>
>> >>> >>> >> Thoughts ?
>> >>> >>> >
>> >>> >>> > Is MongoDB not suitable?
>> >>> >>> > We already include a jar for it.
>> >>> >>> >
>> >>> >>
>> >>> >>
>> >>> >> Mongodb is more document oriented and has another type of use
>> cases.
>> >>> One
>> >>> >> interesting feature of redis is lists where you can pop a line it
>> will
>> >>> not
>> >>> >> be available to next calls, it is suitable for tests that need
>> >>> uniqueness
>> >>> >> accross all nodes.
>> >>> >>
>> >>> >>> >> --
>> >>> >>> >> Regards.
>> >>> >>> >> Philippe M.
>> >>> >>> >> @philmdot <https://twitter.com/philmdot>
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Cordialement.
>> >>> >> Philippe Mouawad.
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >
>> >>> > --
>> >>> > Cordialement.
>> >>> > Philippe Mouawad.
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Cordialement.
>> >> Philippe Mouawad.
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
We could sue Spring-Data but it would introduce a lot of dependencies.
On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
> On 13 November 2013 21:59, Philippe Mouawad <ph...@gmail.com>
> wrote:
> > Hello sebb,
> > Shall I commit this development or do we wait for 2.10.1 to be released ?
>
> I am still concerned that this addition is specific to the Redis server.
> I would much prefer to see a generic solution that can use various
> different kinds of servers.
>
> > When do you plan to commit your devs on keytool ?
>
If I have time I will do it , I plan the following, if you want more let me
know:
- Add keytool path property
- Add popup to be clear about error
- Add info about how to fix it
>
> Sorry, have not been able to spend any time on this recently.
>
> > Thank you
> > Regards
> > Philippe
> >
> >
> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
> philippe.mouawad@gmail.com
> >> wrote:
> >
> >> Hello Sebb,
> >> My answers below.
> >>
> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> >>
> >>> On 2 November 2013 08:28, Philippe Mouawad <philippe.mouawad@gmail.com
> >
> >>> wrote:
> >>> > Hello,
> >>> > Can I proceed with commit these up coming days ?
> >>>
> >>> I'm not sure that the discussion was completed.
> >>>
> >>> As far as I can tell, the proposal only suits some types of
> >>> cloud-based test, and relies on 3rd party servers to hold the data.
> >>>
> >>
> >> No in my opinion if fits many scenarios:
> >> - Cloud based tests, this one seems to me an important one, as Cloud
> based
> >> usage are increasing
> >> - Distributed tests, even if it is possible to do it with CSV, having
> the
> >> data in a remote server is much easier to manage than having to
> >> split/distribute on servers. Even it is true it requires some skills to
> >> manage correctly the Redis server
> >> - Continuous integration tests where also having the data in a
> centralised
> >> , remote servers is easier than managing CSVs. In this case Redis server
> >> plays the same role as a JDBC repository
> >> - Compared to a database it should perform better for high load tests
> >> since it's an in-memory repo (although this can be done in SQL
> databases),
> >> see http://redis.io/topics/benchmarks
> >>
> >>
> >>> I'm not yet convinced that this is how we should be extending JMeter.
> >>>
> >>
> >> I don't understand this argument, it would be another string for JMeter,
> >> my implementation is only 2 classes + properties for I18N ?
> >>
> >> What do you propose in this case ?
> >>
> >>
> >>
> >>> > Thanks
> >>> > Regards
> >>> > Philippe
> >>> >
> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
> >>> >
> >>> >>
> >>> >>
> >>> >> On Monday, October 28, 2013, sebb wrote:
> >>> >>
> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
> >>> philippe.mouawad@gmail.com>
> >>> >>> wrote:
> >>> >>> >> Hello,
> >>> >>> >> These days Cloud based testing is becoming popular and having to
> >>> >>> distribute
> >>> >>> >> test data accross many servers through CSV can become painful if
> >>> not
> >>> >>> >> impossible.
> >>> >>> >>
> >>> >>> >> Even without Cloud, when using distributed testing you always
> have
> >>> to
> >>> >>> >> replicate your data on all servers, which is a painful manual
> step.
> >>> >>> >>
> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for these use
> >>> cases
> >>> >>> ?
> >>> >>> >>
> >>> >>> >> We could do it in many different ways:
> >>> >>> >> - Integrate an automatic CSV replicator, this would remain
> simple
> >>> and
> >>> >>> would
> >>> >>> >> not introduce any tier, but with cloud based it would not
> >>> horizontally
> >>> >>> >> scale easily
> >>> >>> >
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>> > Not sure I follow the scaling argument. The file would only have
> to
> >>> be
> >>> >>> > copied once before the test proper starts.
> >>> >>> > From then on, the data is accessed locally.
> >>> >>> >
> >>> >>
> >>> >>
> >>> >> The scale word was not good, In my thinking the issue is more about
> >>> >> deploying/copying/splitting the file among nodes or cloud members. A
> >>> >> centralized access makes it far easier.
> >>> >>
> >>> >>
> >>> >>> > However, with a database, each node will need at least one
> >>> connection
> >>> >>> > to the database, and every time more data is needed there will be
> >>> >>> > network traffic.
> >>> >>> > Or the database has to be running on the server node.
> >>> >>> >
> >>> >>
> >>> >>
> >>> >> Agree, I was not saying anything different. But as I said this can
> be
> >>> >> useful for middle or low volume
> >>> >>
> >>> >>
> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs
> >>> fine, and
> >>> >>> >> jdbc protocol is not well suited for cloud based deployment
> (But it
> >>> >>> could
> >>> >>> >> also be an interesting feature for Continuous Integration)
> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best choice) ,
> see
> >>> >>> this
> >>> >>> >> high level summary which I find interesting
> >>> >>> >>
> >>> >>>
> >>>
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
> >>> >>> >>
> >>> >>> >> I have implemented a new Redis (based on Java library for Redis)
> >>> >>> DataSet
> >>> >>> >> which I plan to commit if no objection.
> >>> >>> >>
> >>> >>> >> It will introduce 2 new dependencies with Apache License:
> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
> >>> >>> >> - commons-pool
> >>> >>> >
> >>> >>> > There is also a dependency on Redis, but I guess that would not
> be
> >>> >>> bundled.
> >>> >>>
> >>> >>>
> >>> >> no need to anything else than jedis + commons-pool
> >>> >>
> >>> >>
> >>> >>> I've just noticed that Redis is provided as source which needs to
> be
> >>> >>> built before use.
> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
> >>> >>> it's going to be much harder to install Redis.
> >>> >>> In which case additional network traffic will be needed to access
> the
> >>> >>> database.
> >>> >>>
> >>> >>> Also there is no official Windows release.
> >>> >>>
> >>> >>> >> Thoughts ?
> >>> >>> >
> >>> >>> > Is MongoDB not suitable?
> >>> >>> > We already include a jar for it.
> >>> >>> >
> >>> >>
> >>> >>
> >>> >> Mongodb is more document oriented and has another type of use
> cases.
> >>> One
> >>> >> interesting feature of redis is lists where you can pop a line it
> will
> >>> not
> >>> >> be available to next calls, it is suitable for tests that need
> >>> uniqueness
> >>> >> accross all nodes.
> >>> >>
> >>> >>> >> --
> >>> >>> >> Regards.
> >>> >>> >> Philippe M.
> >>> >>> >> @philmdot <https://twitter.com/philmdot>
> >>> >>>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Cordialement.
> >>> >> Philippe Mouawad.
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >
> >>> > --
> >>> > Cordialement.
> >>> > Philippe Mouawad.
> >>>
> >>
> >>
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >>
> >>
> >>
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by sebb <se...@gmail.com>.
On 13 November 2013 21:59, Philippe Mouawad <ph...@gmail.com> wrote:
> Hello sebb,
> Shall I commit this development or do we wait for 2.10.1 to be released ?
I am still concerned that this addition is specific to the Redis server.
I would much prefer to see a generic solution that can use various
different kinds of servers.
> When do you plan to commit your devs on keytool ?
Sorry, have not been able to spend any time on this recently.
> Thank you
> Regards
> Philippe
>
>
> On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <philippe.mouawad@gmail.com
>> wrote:
>
>> Hello Sebb,
>> My answers below.
>>
>> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
>>
>>> On 2 November 2013 08:28, Philippe Mouawad <ph...@gmail.com>
>>> wrote:
>>> > Hello,
>>> > Can I proceed with commit these up coming days ?
>>>
>>> I'm not sure that the discussion was completed.
>>>
>>> As far as I can tell, the proposal only suits some types of
>>> cloud-based test, and relies on 3rd party servers to hold the data.
>>>
>>
>> No in my opinion if fits many scenarios:
>> - Cloud based tests, this one seems to me an important one, as Cloud based
>> usage are increasing
>> - Distributed tests, even if it is possible to do it with CSV, having the
>> data in a remote server is much easier to manage than having to
>> split/distribute on servers. Even it is true it requires some skills to
>> manage correctly the Redis server
>> - Continuous integration tests where also having the data in a centralised
>> , remote servers is easier than managing CSVs. In this case Redis server
>> plays the same role as a JDBC repository
>> - Compared to a database it should perform better for high load tests
>> since it's an in-memory repo (although this can be done in SQL databases),
>> see http://redis.io/topics/benchmarks
>>
>>
>>> I'm not yet convinced that this is how we should be extending JMeter.
>>>
>>
>> I don't understand this argument, it would be another string for JMeter,
>> my implementation is only 2 classes + properties for I18N ?
>>
>> What do you propose in this case ?
>>
>>
>>
>>> > Thanks
>>> > Regards
>>> > Philippe
>>> >
>>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
>>> >
>>> >>
>>> >>
>>> >> On Monday, October 28, 2013, sebb wrote:
>>> >>
>>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
>>> philippe.mouawad@gmail.com>
>>> >>> wrote:
>>> >>> >> Hello,
>>> >>> >> These days Cloud based testing is becoming popular and having to
>>> >>> distribute
>>> >>> >> test data accross many servers through CSV can become painful if
>>> not
>>> >>> >> impossible.
>>> >>> >>
>>> >>> >> Even without Cloud, when using distributed testing you always have
>>> to
>>> >>> >> replicate your data on all servers, which is a painful manual step.
>>> >>> >>
>>> >>> >> Shouldn't we introduce a new DataSet more suitable for these use
>>> cases
>>> >>> ?
>>> >>> >>
>>> >>> >> We could do it in many different ways:
>>> >>> >> - Integrate an automatic CSV replicator, this would remain simple
>>> and
>>> >>> would
>>> >>> >> not introduce any tier, but with cloud based it would not
>>> horizontally
>>> >>> >> scale easily
>>> >>> >
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>> > Not sure I follow the scaling argument. The file would only have to
>>> be
>>> >>> > copied once before the test proper starts.
>>> >>> > From then on, the data is accessed locally.
>>> >>> >
>>> >>
>>> >>
>>> >> The scale word was not good, In my thinking the issue is more about
>>> >> deploying/copying/splitting the file among nodes or cloud members. A
>>> >> centralized access makes it far easier.
>>> >>
>>> >>
>>> >>> > However, with a database, each node will need at least one
>>> connection
>>> >>> > to the database, and every time more data is needed there will be
>>> >>> > network traffic.
>>> >>> > Or the database has to be running on the server node.
>>> >>> >
>>> >>
>>> >>
>>> >> Agree, I was not saying anything different. But as I said this can be
>>> >> useful for middle or low volume
>>> >>
>>> >>
>>> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs
>>> fine, and
>>> >>> >> jdbc protocol is not well suited for cloud based deployment (But it
>>> >>> could
>>> >>> >> also be an interesting feature for Continuous Integration)
>>> >>> >> - Use a NOSQL repository (Redis seems to me the best choice) , see
>>> >>> this
>>> >>> >> high level summary which I find interesting
>>> >>> >>
>>> >>>
>>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>>> >>> >>
>>> >>> >> I have implemented a new Redis (based on Java library for Redis)
>>> >>> DataSet
>>> >>> >> which I plan to commit if no objection.
>>> >>> >>
>>> >>> >> It will introduce 2 new dependencies with Apache License:
>>> >>> >> - Jedis (http://code.google.com/p/jedis/)
>>> >>> >> - commons-pool
>>> >>> >
>>> >>> > There is also a dependency on Redis, but I guess that would not be
>>> >>> bundled.
>>> >>>
>>> >>>
>>> >> no need to anything else than jedis + commons-pool
>>> >>
>>> >>
>>> >>> I've just noticed that Redis is provided as source which needs to be
>>> >>> built before use.
>>> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>>> >>> it's going to be much harder to install Redis.
>>> >>> In which case additional network traffic will be needed to access the
>>> >>> database.
>>> >>>
>>> >>> Also there is no official Windows release.
>>> >>>
>>> >>> >> Thoughts ?
>>> >>> >
>>> >>> > Is MongoDB not suitable?
>>> >>> > We already include a jar for it.
>>> >>> >
>>> >>
>>> >>
>>> >> Mongodb is more document oriented and has another type of use cases.
>>> One
>>> >> interesting feature of redis is lists where you can pop a line it will
>>> not
>>> >> be available to next calls, it is suitable for tests that need
>>> uniqueness
>>> >> accross all nodes.
>>> >>
>>> >>> >> --
>>> >>> >> Regards.
>>> >>> >> Philippe M.
>>> >>> >> @philmdot <https://twitter.com/philmdot>
>>> >>>
>>> >>
>>> >>
>>> >> --
>>> >> Cordialement.
>>> >> Philippe Mouawad.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >
>>> > --
>>> > Cordialement.
>>> > Philippe Mouawad.
>>>
>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello sebb,
Shall I commit this development or do we wait for 2.10.1 to be released ?
When do you plan to commit your devs on keytool ?
Thank you
Regards
Philippe
On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <philippe.mouawad@gmail.com
> wrote:
> Hello Sebb,
> My answers below.
>
> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
>
>> On 2 November 2013 08:28, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>> > Hello,
>> > Can I proceed with commit these up coming days ?
>>
>> I'm not sure that the discussion was completed.
>>
>> As far as I can tell, the proposal only suits some types of
>> cloud-based test, and relies on 3rd party servers to hold the data.
>>
>
> No in my opinion if fits many scenarios:
> - Cloud based tests, this one seems to me an important one, as Cloud based
> usage are increasing
> - Distributed tests, even if it is possible to do it with CSV, having the
> data in a remote server is much easier to manage than having to
> split/distribute on servers. Even it is true it requires some skills to
> manage correctly the Redis server
> - Continuous integration tests where also having the data in a centralised
> , remote servers is easier than managing CSVs. In this case Redis server
> plays the same role as a JDBC repository
> - Compared to a database it should perform better for high load tests
> since it's an in-memory repo (although this can be done in SQL databases),
> see http://redis.io/topics/benchmarks
>
>
>> I'm not yet convinced that this is how we should be extending JMeter.
>>
>
> I don't understand this argument, it would be another string for JMeter,
> my implementation is only 2 classes + properties for I18N ?
>
> What do you propose in this case ?
>
>
>
>> > Thanks
>> > Regards
>> > Philippe
>> >
>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
>> >
>> >>
>> >>
>> >> On Monday, October 28, 2013, sebb wrote:
>> >>
>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
>> philippe.mouawad@gmail.com>
>> >>> wrote:
>> >>> >> Hello,
>> >>> >> These days Cloud based testing is becoming popular and having to
>> >>> distribute
>> >>> >> test data accross many servers through CSV can become painful if
>> not
>> >>> >> impossible.
>> >>> >>
>> >>> >> Even without Cloud, when using distributed testing you always have
>> to
>> >>> >> replicate your data on all servers, which is a painful manual step.
>> >>> >>
>> >>> >> Shouldn't we introduce a new DataSet more suitable for these use
>> cases
>> >>> ?
>> >>> >>
>> >>> >> We could do it in many different ways:
>> >>> >> - Integrate an automatic CSV replicator, this would remain simple
>> and
>> >>> would
>> >>> >> not introduce any tier, but with cloud based it would not
>> horizontally
>> >>> >> scale easily
>> >>> >
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>> > Not sure I follow the scaling argument. The file would only have to
>> be
>> >>> > copied once before the test proper starts.
>> >>> > From then on, the data is accessed locally.
>> >>> >
>> >>
>> >>
>> >> The scale word was not good, In my thinking the issue is more about
>> >> deploying/copying/splitting the file among nodes or cloud members. A
>> >> centralized access makes it far easier.
>> >>
>> >>
>> >>> > However, with a database, each node will need at least one
>> connection
>> >>> > to the database, and every time more data is needed there will be
>> >>> > network traffic.
>> >>> > Or the database has to be running on the server node.
>> >>> >
>> >>
>> >>
>> >> Agree, I was not saying anything different. But as I said this can be
>> >> useful for middle or low volume
>> >>
>> >>
>> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs
>> fine, and
>> >>> >> jdbc protocol is not well suited for cloud based deployment (But it
>> >>> could
>> >>> >> also be an interesting feature for Continuous Integration)
>> >>> >> - Use a NOSQL repository (Redis seems to me the best choice) , see
>> >>> this
>> >>> >> high level summary which I find interesting
>> >>> >>
>> >>>
>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>> >>> >>
>> >>> >> I have implemented a new Redis (based on Java library for Redis)
>> >>> DataSet
>> >>> >> which I plan to commit if no objection.
>> >>> >>
>> >>> >> It will introduce 2 new dependencies with Apache License:
>> >>> >> - Jedis (http://code.google.com/p/jedis/)
>> >>> >> - commons-pool
>> >>> >
>> >>> > There is also a dependency on Redis, but I guess that would not be
>> >>> bundled.
>> >>>
>> >>>
>> >> no need to anything else than jedis + commons-pool
>> >>
>> >>
>> >>> I've just noticed that Redis is provided as source which needs to be
>> >>> built before use.
>> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>> >>> it's going to be much harder to install Redis.
>> >>> In which case additional network traffic will be needed to access the
>> >>> database.
>> >>>
>> >>> Also there is no official Windows release.
>> >>>
>> >>> >> Thoughts ?
>> >>> >
>> >>> > Is MongoDB not suitable?
>> >>> > We already include a jar for it.
>> >>> >
>> >>
>> >>
>> >> Mongodb is more document oriented and has another type of use cases.
>> One
>> >> interesting feature of redis is lists where you can pop a line it will
>> not
>> >> be available to next calls, it is suitable for tests that need
>> uniqueness
>> >> accross all nodes.
>> >>
>> >>> >> --
>> >>> >> Regards.
>> >>> >> Philippe M.
>> >>> >> @philmdot <https://twitter.com/philmdot>
>> >>>
>> >>
>> >>
>> >> --
>> >> Cordialement.
>> >> Philippe Mouawad.
>> >>
>> >>
>> >>
>> >>
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello Sebb,
My answers below.
On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> On 2 November 2013 08:28, Philippe Mouawad <ph...@gmail.com>
> wrote:
> > Hello,
> > Can I proceed with commit these up coming days ?
>
> I'm not sure that the discussion was completed.
>
> As far as I can tell, the proposal only suits some types of
> cloud-based test, and relies on 3rd party servers to hold the data.
>
No in my opinion if fits many scenarios:
- Cloud based tests, this one seems to me an important one, as Cloud based
usage are increasing
- Distributed tests, even if it is possible to do it with CSV, having the
data in a remote server is much easier to manage than having to
split/distribute on servers. Even it is true it requires some skills to
manage correctly the Redis server
- Continuous integration tests where also having the data in a centralised
, remote servers is easier than managing CSVs. In this case Redis server
plays the same role as a JDBC repository
- Compared to a database it should perform better for high load tests since
it's an in-memory repo (although this can be done in SQL databases), see
http://redis.io/topics/benchmarks
> I'm not yet convinced that this is how we should be extending JMeter.
>
I don't understand this argument, it would be another string for JMeter, my
implementation is only 2 classes + properties for I18N ?
What do you propose in this case ?
> > Thanks
> > Regards
> > Philippe
> >
> > On Monday, October 28, 2013, Philippe Mouawad wrote:
> >
> >>
> >>
> >> On Monday, October 28, 2013, sebb wrote:
> >>
> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
> >>> > On 26 October 2013 20:23, Philippe Mouawad <
> philippe.mouawad@gmail.com>
> >>> wrote:
> >>> >> Hello,
> >>> >> These days Cloud based testing is becoming popular and having to
> >>> distribute
> >>> >> test data accross many servers through CSV can become painful if not
> >>> >> impossible.
> >>> >>
> >>> >> Even without Cloud, when using distributed testing you always have
> to
> >>> >> replicate your data on all servers, which is a painful manual step.
> >>> >>
> >>> >> Shouldn't we introduce a new DataSet more suitable for these use
> cases
> >>> ?
> >>> >>
> >>> >> We could do it in many different ways:
> >>> >> - Integrate an automatic CSV replicator, this would remain simple
> and
> >>> would
> >>> >> not introduce any tier, but with cloud based it would not
> horizontally
> >>> >> scale easily
> >>> >
> >>
> >>
> >>
> >>
> >>
> >>> > Not sure I follow the scaling argument. The file would only have to
> be
> >>> > copied once before the test proper starts.
> >>> > From then on, the data is accessed locally.
> >>> >
> >>
> >>
> >> The scale word was not good, In my thinking the issue is more about
> >> deploying/copying/splitting the file among nodes or cloud members. A
> >> centralized access makes it far easier.
> >>
> >>
> >>> > However, with a database, each node will need at least one connection
> >>> > to the database, and every time more data is needed there will be
> >>> > network traffic.
> >>> > Or the database has to be running on the server node.
> >>> >
> >>
> >>
> >> Agree, I was not saying anything different. But as I said this can be
> >> useful for middle or low volume
> >>
> >>
> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs fine,
> and
> >>> >> jdbc protocol is not well suited for cloud based deployment (But it
> >>> could
> >>> >> also be an interesting feature for Continuous Integration)
> >>> >> - Use a NOSQL repository (Redis seems to me the best choice) , see
> >>> this
> >>> >> high level summary which I find interesting
> >>> >>
> >>>
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
> >>> >>
> >>> >> I have implemented a new Redis (based on Java library for Redis)
> >>> DataSet
> >>> >> which I plan to commit if no objection.
> >>> >>
> >>> >> It will introduce 2 new dependencies with Apache License:
> >>> >> - Jedis (http://code.google.com/p/jedis/)
> >>> >> - commons-pool
> >>> >
> >>> > There is also a dependency on Redis, but I guess that would not be
> >>> bundled.
> >>>
> >>>
> >> no need to anything else than jedis + commons-pool
> >>
> >>
> >>> I've just noticed that Redis is provided as source which needs to be
> >>> built before use.
> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
> >>> it's going to be much harder to install Redis.
> >>> In which case additional network traffic will be needed to access the
> >>> database.
> >>>
> >>> Also there is no official Windows release.
> >>>
> >>> >> Thoughts ?
> >>> >
> >>> > Is MongoDB not suitable?
> >>> > We already include a jar for it.
> >>> >
> >>
> >>
> >> Mongodb is more document oriented and has another type of use cases.
> One
> >> interesting feature of redis is lists where you can pop a line it will
> not
> >> be available to next calls, it is suitable for tests that need
> uniqueness
> >> accross all nodes.
> >>
> >>> >> --
> >>> >> Regards.
> >>> >> Philippe M.
> >>> >> @philmdot <https://twitter.com/philmdot>
> >>>
> >>
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >>
> >>
> >>
> >>
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by sebb <se...@gmail.com>.
On 2 November 2013 08:28, Philippe Mouawad <ph...@gmail.com> wrote:
> Hello,
> Can I proceed with commit these up coming days ?
I'm not sure that the discussion was completed.
As far as I can tell, the proposal only suits some types of
cloud-based test, and relies on 3rd party servers to hold the data.
I'm not yet convinced that this is how we should be extending JMeter.
> Thanks
> Regards
> Philippe
>
> On Monday, October 28, 2013, Philippe Mouawad wrote:
>
>>
>>
>> On Monday, October 28, 2013, sebb wrote:
>>
>>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>>> > On 26 October 2013 20:23, Philippe Mouawad <ph...@gmail.com>
>>> wrote:
>>> >> Hello,
>>> >> These days Cloud based testing is becoming popular and having to
>>> distribute
>>> >> test data accross many servers through CSV can become painful if not
>>> >> impossible.
>>> >>
>>> >> Even without Cloud, when using distributed testing you always have to
>>> >> replicate your data on all servers, which is a painful manual step.
>>> >>
>>> >> Shouldn't we introduce a new DataSet more suitable for these use cases
>>> ?
>>> >>
>>> >> We could do it in many different ways:
>>> >> - Integrate an automatic CSV replicator, this would remain simple and
>>> would
>>> >> not introduce any tier, but with cloud based it would not horizontally
>>> >> scale easily
>>> >
>>
>>
>>
>>
>>
>>> > Not sure I follow the scaling argument. The file would only have to be
>>> > copied once before the test proper starts.
>>> > From then on, the data is accessed locally.
>>> >
>>
>>
>> The scale word was not good, In my thinking the issue is more about
>> deploying/copying/splitting the file among nodes or cloud members. A
>> centralized access makes it far easier.
>>
>>
>>> > However, with a database, each node will need at least one connection
>>> > to the database, and every time more data is needed there will be
>>> > network traffic.
>>> > Or the database has to be running on the server node.
>>> >
>>
>>
>> Agree, I was not saying anything different. But as I said this can be
>> useful for middle or low volume
>>
>>
>>> >> - Use a JDBC DataSet, but we would need to ensure it performs fine, and
>>> >> jdbc protocol is not well suited for cloud based deployment (But it
>>> could
>>> >> also be an interesting feature for Continuous Integration)
>>> >> - Use a NOSQL repository (Redis seems to me the best choice) , see
>>> this
>>> >> high level summary which I find interesting
>>> >>
>>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>>> >>
>>> >> I have implemented a new Redis (based on Java library for Redis)
>>> DataSet
>>> >> which I plan to commit if no objection.
>>> >>
>>> >> It will introduce 2 new dependencies with Apache License:
>>> >> - Jedis (http://code.google.com/p/jedis/)
>>> >> - commons-pool
>>> >
>>> > There is also a dependency on Redis, but I guess that would not be
>>> bundled.
>>>
>>>
>> no need to anything else than jedis + commons-pool
>>
>>
>>> I've just noticed that Redis is provided as source which needs to be
>>> built before use.
>>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>>> it's going to be much harder to install Redis.
>>> In which case additional network traffic will be needed to access the
>>> database.
>>>
>>> Also there is no official Windows release.
>>>
>>> >> Thoughts ?
>>> >
>>> > Is MongoDB not suitable?
>>> > We already include a jar for it.
>>> >
>>
>>
>> Mongodb is more document oriented and has another type of use cases. One
>> interesting feature of redis is lists where you can pop a line it will not
>> be available to next calls, it is suitable for tests that need uniqueness
>> accross all nodes.
>>
>>> >> --
>>> >> Regards.
>>> >> Philippe M.
>>> >> @philmdot <https://twitter.com/philmdot>
>>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
Can I proceed with commit these up coming days ?
Thanks
Regards
Philippe
On Monday, October 28, 2013, Philippe Mouawad wrote:
>
>
> On Monday, October 28, 2013, sebb wrote:
>
>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>> > On 26 October 2013 20:23, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>> >> Hello,
>> >> These days Cloud based testing is becoming popular and having to
>> distribute
>> >> test data accross many servers through CSV can become painful if not
>> >> impossible.
>> >>
>> >> Even without Cloud, when using distributed testing you always have to
>> >> replicate your data on all servers, which is a painful manual step.
>> >>
>> >> Shouldn't we introduce a new DataSet more suitable for these use cases
>> ?
>> >>
>> >> We could do it in many different ways:
>> >> - Integrate an automatic CSV replicator, this would remain simple and
>> would
>> >> not introduce any tier, but with cloud based it would not horizontally
>> >> scale easily
>> >
>
>
>
>
>
>> > Not sure I follow the scaling argument. The file would only have to be
>> > copied once before the test proper starts.
>> > From then on, the data is accessed locally.
>> >
>
>
> The scale word was not good, In my thinking the issue is more about
> deploying/copying/splitting the file among nodes or cloud members. A
> centralized access makes it far easier.
>
>
>> > However, with a database, each node will need at least one connection
>> > to the database, and every time more data is needed there will be
>> > network traffic.
>> > Or the database has to be running on the server node.
>> >
>
>
> Agree, I was not saying anything different. But as I said this can be
> useful for middle or low volume
>
>
>> >> - Use a JDBC DataSet, but we would need to ensure it performs fine, and
>> >> jdbc protocol is not well suited for cloud based deployment (But it
>> could
>> >> also be an interesting feature for Continuous Integration)
>> >> - Use a NOSQL repository (Redis seems to me the best choice) , see
>> this
>> >> high level summary which I find interesting
>> >>
>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>> >>
>> >> I have implemented a new Redis (based on Java library for Redis)
>> DataSet
>> >> which I plan to commit if no objection.
>> >>
>> >> It will introduce 2 new dependencies with Apache License:
>> >> - Jedis (http://code.google.com/p/jedis/)
>> >> - commons-pool
>> >
>> > There is also a dependency on Redis, but I guess that would not be
>> bundled.
>>
>>
> no need to anything else than jedis + commons-pool
>
>
>> I've just noticed that Redis is provided as source which needs to be
>> built before use.
>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>> it's going to be much harder to install Redis.
>> In which case additional network traffic will be needed to access the
>> database.
>>
>> Also there is no official Windows release.
>>
>> >> Thoughts ?
>> >
>> > Is MongoDB not suitable?
>> > We already include a jar for it.
>> >
>
>
> Mongodb is more document oriented and has another type of use cases. One
> interesting feature of redis is lists where you can pop a line it will not
> be available to next calls, it is suitable for tests that need uniqueness
> accross all nodes.
>
>> >> --
>> >> Regards.
>> >> Philippe M.
>> >> @philmdot <https://twitter.com/philmdot>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
On Monday, October 28, 2013, sebb wrote:
> On 28 October 2013 01:26, sebb <sebbaz@gmail.com <javascript:;>> wrote:
> > On 26 October 2013 20:23, Philippe Mouawad <philippe.mouawad@gmail.com<javascript:;>>
> wrote:
> >> Hello,
> >> These days Cloud based testing is becoming popular and having to
> distribute
> >> test data accross many servers through CSV can become painful if not
> >> impossible.
> >>
> >> Even without Cloud, when using distributed testing you always have to
> >> replicate your data on all servers, which is a painful manual step.
> >>
> >> Shouldn't we introduce a new DataSet more suitable for these use cases ?
> >>
> >> We could do it in many different ways:
> >> - Integrate an automatic CSV replicator, this would remain simple and
> would
> >> not introduce any tier, but with cloud based it would not horizontally
> >> scale easily
> >
> > Not sure I follow the scaling argument. The file would only have to be
> > copied once before the test proper starts.
> > From then on, the data is accessed locally.
> >
The scale word was not good, In my thinking the issue is more about
deploying/copying/splitting the file among nodes or cloud members. A
centralized access makes it far easier.
> > However, with a database, each node will need at least one connection
> > to the database, and every time more data is needed there will be
> > network traffic.
> > Or the database has to be running on the server node.
> >
Agree, I was not saying anything different. But as I said this can be
useful for middle or low volume
> >> - Use a JDBC DataSet, but we would need to ensure it performs fine, and
> >> jdbc protocol is not well suited for cloud based deployment (But it
> could
> >> also be an interesting feature for Continuous Integration)
> >> - Use a NOSQL repository (Redis seems to me the best choice) , see this
> >> high level summary which I find interesting
> >>
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
> >>
> >> I have implemented a new Redis (based on Java library for Redis) DataSet
> >> which I plan to commit if no objection.
> >>
> >> It will introduce 2 new dependencies with Apache License:
> >> - Jedis (http://code.google.com/p/jedis/)
> >> - commons-pool
> >
> > There is also a dependency on Redis, but I guess that would not be
> bundled.
>
>
no need to anything else than jedis + commons-pool
> I've just noticed that Redis is provided as source which needs to be
> built before use.
> If it's difficult to copy CSV files to cloud nodes, it seems to me
> it's going to be much harder to install Redis.
> In which case additional network traffic will be needed to access the
> database.
>
> Also there is no official Windows release.
>
> >> Thoughts ?
> >
> > Is MongoDB not suitable?
> > We already include a jar for it.
> >
Mongodb is more document oriented and has another type of use cases. One
interesting feature of redis is lists where you can pop a line it will not
be available to next calls, it is suitable for tests that need uniqueness
accross all nodes.
> >> --
> >> Regards.
> >> Philippe M.
> >> @philmdot <https://twitter.com/philmdot>
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by sebb <se...@gmail.com>.
On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
> On 26 October 2013 20:23, Philippe Mouawad <ph...@gmail.com> wrote:
>> Hello,
>> These days Cloud based testing is becoming popular and having to distribute
>> test data accross many servers through CSV can become painful if not
>> impossible.
>>
>> Even without Cloud, when using distributed testing you always have to
>> replicate your data on all servers, which is a painful manual step.
>>
>> Shouldn't we introduce a new DataSet more suitable for these use cases ?
>>
>> We could do it in many different ways:
>> - Integrate an automatic CSV replicator, this would remain simple and would
>> not introduce any tier, but with cloud based it would not horizontally
>> scale easily
>
> Not sure I follow the scaling argument. The file would only have to be
> copied once before the test proper starts.
> From then on, the data is accessed locally.
>
> However, with a database, each node will need at least one connection
> to the database, and every time more data is needed there will be
> network traffic.
> Or the database has to be running on the server node.
>
>> - Use a JDBC DataSet, but we would need to ensure it performs fine, and
>> jdbc protocol is not well suited for cloud based deployment (But it could
>> also be an interesting feature for Continuous Integration)
>> - Use a NOSQL repository (Redis seems to me the best choice) , see this
>> high level summary which I find interesting
>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>>
>> I have implemented a new Redis (based on Java library for Redis) DataSet
>> which I plan to commit if no objection.
>>
>> It will introduce 2 new dependencies with Apache License:
>> - Jedis (http://code.google.com/p/jedis/)
>> - commons-pool
>
> There is also a dependency on Redis, but I guess that would not be bundled.
I've just noticed that Redis is provided as source which needs to be
built before use.
If it's difficult to copy CSV files to cloud nodes, it seems to me
it's going to be much harder to install Redis.
In which case additional network traffic will be needed to access the database.
Also there is no official Windows release.
>> Thoughts ?
>
> Is MongoDB not suitable?
> We already include a jar for it.
>
>> --
>> Regards.
>> Philippe M.
>> @philmdot <https://twitter.com/philmdot>
Re: New DataSet implementation
Posted by sebb <se...@gmail.com>.
On 26 October 2013 20:23, Philippe Mouawad <ph...@gmail.com> wrote:
> Hello,
> These days Cloud based testing is becoming popular and having to distribute
> test data accross many servers through CSV can become painful if not
> impossible.
>
> Even without Cloud, when using distributed testing you always have to
> replicate your data on all servers, which is a painful manual step.
>
> Shouldn't we introduce a new DataSet more suitable for these use cases ?
>
> We could do it in many different ways:
> - Integrate an automatic CSV replicator, this would remain simple and would
> not introduce any tier, but with cloud based it would not horizontally
> scale easily
Not sure I follow the scaling argument. The file would only have to be
copied once before the test proper starts.
>From then on, the data is accessed locally.
However, with a database, each node will need at least one connection
to the database, and every time more data is needed there will be
network traffic.
Or the database has to be running on the server node.
> - Use a JDBC DataSet, but we would need to ensure it performs fine, and
> jdbc protocol is not well suited for cloud based deployment (But it could
> also be an interesting feature for Continuous Integration)
> - Use a NOSQL repository (Redis seems to me the best choice) , see this
> high level summary which I find interesting
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>
> I have implemented a new Redis (based on Java library for Redis) DataSet
> which I plan to commit if no objection.
>
> It will introduce 2 new dependencies with Apache License:
> - Jedis (http://code.google.com/p/jedis/)
> - commons-pool
There is also a dependency on Redis, but I guess that would not be bundled.
> Thoughts ?
Is MongoDB not suitable?
We already include a jar for it.
> --
> Regards.
> Philippe M.
> @philmdot <https://twitter.com/philmdot>
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
It will be the default behaviour.
On Sat, Oct 26, 2013 at 10:07 PM, Shmuel Krakower <sh...@gmail.com>wrote:
> Without getting to the techi aspect, sounds good, it's about time to have
> centralized datasets, so +1.
>
> Does your implementation also allow to choose sharing mode between the load
> generators?
> I.e. if I want a single value to be used only once during a test, will your
> implementation support it, without this value being used more than once
> across the load generators?
>
> Shmuel Krakower.
> www.Beatsoo.org - re-use your jmeter scripts for application performance
> monitoring from worldwide locations for free.
>
>
> On Sat, Oct 26, 2013 at 10:23 PM, Philippe Mouawad <
> philippe.mouawad@gmail.com> wrote:
>
> > Hello,
> > These days Cloud based testing is becoming popular and having to
> distribute
> > test data accross many servers through CSV can become painful if not
> > impossible.
> > Even without Cloud, when using distributed testing you always have to
> > replicate your data on all servers, which is a painful manual step.
> >
> > Shouldn't we introduce a new DataSet more suitable for these use cases ?
> >
> > We could do it in many different ways:
> > - Integrate an automatic CSV replicator, this would remain simple and
> would
> > not introduce any tier, but with cloud based it would not horizontally
> > scale easily
> > - Use a JDBC DataSet, but we would need to ensure it performs fine, and
> > jdbc protocol is not well suited for cloud based deployment (But it could
> > also be an interesting feature for Continuous Integration)
> > - Use a NOSQL repository (Redis seems to me the best choice) , see this
> > high level summary which I find interesting
> >
> >
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
> >
> > I have implemented a new Redis (based on Java library for Redis) DataSet
> > which I plan to commit if no objection.
> >
> > It will introduce 2 new dependencies with Apache License:
> > - Jedis (http://code.google.com/p/jedis/)
> > - commons-pool
> >
> > Thoughts ?
> > --
> > Regards.
> > Philippe M.
> > @philmdot <https://twitter.com/philmdot>
> >
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by Shmuel Krakower <sh...@gmail.com>.
Without getting to the techi aspect, sounds good, it's about time to have
centralized datasets, so +1.
Does your implementation also allow to choose sharing mode between the load
generators?
I.e. if I want a single value to be used only once during a test, will your
implementation support it, without this value being used more than once
across the load generators?
Shmuel Krakower.
www.Beatsoo.org - re-use your jmeter scripts for application performance
monitoring from worldwide locations for free.
On Sat, Oct 26, 2013 at 10:23 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:
> Hello,
> These days Cloud based testing is becoming popular and having to distribute
> test data accross many servers through CSV can become painful if not
> impossible.
> Even without Cloud, when using distributed testing you always have to
> replicate your data on all servers, which is a painful manual step.
>
> Shouldn't we introduce a new DataSet more suitable for these use cases ?
>
> We could do it in many different ways:
> - Integrate an automatic CSV replicator, this would remain simple and would
> not introduce any tier, but with cloud based it would not horizontally
> scale easily
> - Use a JDBC DataSet, but we would need to ensure it performs fine, and
> jdbc protocol is not well suited for cloud based deployment (But it could
> also be an interesting feature for Continuous Integration)
> - Use a NOSQL repository (Redis seems to me the best choice) , see this
> high level summary which I find interesting
>
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>
> I have implemented a new Redis (based on Java library for Redis) DataSet
> which I plan to commit if no objection.
>
> It will introduce 2 new dependencies with Apache License:
> - Jedis (http://code.google.com/p/jedis/)
> - commons-pool
>
> Thoughts ?
> --
> Regards.
> Philippe M.
> @philmdot <https://twitter.com/philmdot>
>