You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jmeter.apache.org by Philippe Mouawad <ph...@gmail.com> on 2013/11/02 09:28:26 UTC
Re: New DataSet implementation
Hello,
Can I proceed with commit these up coming days ?
Thanks
Regards
Philippe
On Monday, October 28, 2013, Philippe Mouawad wrote:
>
>
> On Monday, October 28, 2013, sebb wrote:
>
>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>> > On 26 October 2013 20:23, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>> >> Hello,
>> >> These days Cloud based testing is becoming popular and having to
>> distribute
>> >> test data accross many servers through CSV can become painful if not
>> >> impossible.
>> >>
>> >> Even without Cloud, when using distributed testing you always have to
>> >> replicate your data on all servers, which is a painful manual step.
>> >>
>> >> Shouldn't we introduce a new DataSet more suitable for these use cases
>> ?
>> >>
>> >> We could do it in many different ways:
>> >> - Integrate an automatic CSV replicator, this would remain simple and
>> would
>> >> not introduce any tier, but with cloud based it would not horizontally
>> >> scale easily
>> >
>
>
>
>
>
>> > Not sure I follow the scaling argument. The file would only have to be
>> > copied once before the test proper starts.
>> > From then on, the data is accessed locally.
>> >
>
>
> The scale word was not good, In my thinking the issue is more about
> deploying/copying/splitting the file among nodes or cloud members. A
> centralized access makes it far easier.
>
>
>> > However, with a database, each node will need at least one connection
>> > to the database, and every time more data is needed there will be
>> > network traffic.
>> > Or the database has to be running on the server node.
>> >
>
>
> Agree, I was not saying anything different. But as I said this can be
> useful for middle or low volume
>
>
>> >> - Use a JDBC DataSet, but we would need to ensure it performs fine, and
>> >> jdbc protocol is not well suited for cloud based deployment (But it
>> could
>> >> also be an interesting feature for Continuous Integration)
>> >> - Use a NOSQL repository (Redis seems to me the best choice) , see
>> this
>> >> high level summary which I find interesting
>> >>
>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>> >>
>> >> I have implemented a new Redis (based on Java library for Redis)
>> DataSet
>> >> which I plan to commit if no objection.
>> >>
>> >> It will introduce 2 new dependencies with Apache License:
>> >> - Jedis (http://code.google.com/p/jedis/)
>> >> - commons-pool
>> >
>> > There is also a dependency on Redis, but I guess that would not be
>> bundled.
>>
>>
> no need to anything else than jedis + commons-pool
>
>
>> I've just noticed that Redis is provided as source which needs to be
>> built before use.
>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>> it's going to be much harder to install Redis.
>> In which case additional network traffic will be needed to access the
>> database.
>>
>> Also there is no official Windows release.
>>
>> >> Thoughts ?
>> >
>> > Is MongoDB not suitable?
>> > We already include a jar for it.
>> >
>
>
> Mongodb is more document oriented and has another type of use cases. One
> interesting feature of redis is lists where you can pop a line it will not
> be available to next calls, it is suitable for tests that need uniqueness
> accross all nodes.
>
>> >> --
>> >> Regards.
>> >> Philippe M.
>> >> @philmdot <https://twitter.com/philmdot>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
For information, as we didn't get consensus on commiting the piece of code
to JMeter core and I didn't want to throw away the time I spent
implementing it I have contributed it to third party jmeter-plugins
project, it should be in next version of the plugins.
https://github.com/undera/jmeter-plugins/pull/22
I still think it would have been nice as is in core, creating a common
gui/api for cloud based libs being a bit complex as config properties
really differ as long as purposes.
Regards
Philippe M.
On Thursday, December 19, 2013, Philippe Mouawad wrote:
>
>
>
> On Sat, Nov 30, 2013 at 10:52 AM, sebb <sebbaz@gmail.com<javascript:_e({}, 'cvml', 'sebbaz@gmail.com');>
> > wrote:
>
>> On 27 November 2013 21:21, Philippe Mouawad <philippe.mouawad@gmail.com<javascript:_e({}, 'cvml', 'philippe.mouawad@gmail.com');>>
>> wrote:
>> > On Tue, Nov 26, 2013 at 1:05 AM, sebb <sebbaz@gmail.com<javascript:_e({}, 'cvml', 'sebbaz@gmail.com');>>
>> wrote:
>> >
>> >> On 23 November 2013 13:48, Philippe Mouawad <
>> philippe.mouawad@gmail.com <javascript:_e({}, 'cvml',
>> 'philippe.mouawad@gmail.com');>>
>> >> wrote:
>> >> > Hello sebb,
>> >> > What's your decision on this point ?
>> >>
>> >> I don't make the decisions - they are ideally arrived at by consensus;
>> >> failing that, majority.
>> >>
>> >> > Still having objection to code commit ?
>> >>
>> >> Yes, because it seems wrong to include code for a specific database
>> >> implementation.
>> >> We should be implementing generic solutions.
>> >>
>> >> So would you be ok to go for Spring-data based implementation ?
>> > It will impact number of embedded libraries.
>>
>> There are other possibilities for NoSQL as well, for example
>> http://appscale.cs.ucsb.edu/datastores.html - AppScale
>>
> Commercial no ?
>
>> http://thrift.apache.org/ - Apache Thrift
>>
>
> Didn't dig into it yet, but could you point me to a way to do it if you
> already did ?
>
>>
>> See also
>>
>> http://www.oracle.com/technetwork/database/nosqldb/overview/nosql-api-497225.html
>>
>> http://stackoverflow.com/questions/9252034/is-there-an-emerging-standard-api-for-nosql-databases
>>
>> http://stackoverflow.com/questions/3439878/are-there-any-nosql-standards-emerging
>>
>> Given that there is no standard like JDBC available (and there may
>> never be), what I think needs to happen is to decide where in JMeter
>> such databases may be useful (probably not only CSV Dataset) and then
>> define a suitable API for this.
>> Users can then implement the API in terms of whatever database they
>> wish to use (which might include a JDBC implementation).
>> This is not something to be rushed into.
>>
>
> I don't totally agree with you, I prefer sometimes that we move forward
> with "non perfect" thing and improve it in next steps.
>
> Also I am a bit annoyed that time I spent on this development is kind of
> thrown away while it is really easy to use and have tested it rather
> thoroughly.
>
>
>>
>> In any case, whatever is decided on belongs in a later version of JMeter.
>> We first need to get out the version with various fixes to keytool etc.
>>
>
> Dev is ready and could have been commited but would have agreed to wait.
>
>
> >
> >> > Thanks
> >> > Regards
> >> > Philippe
> >> >
> >> >
> >> > On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
> >> > philippe.mouawad@gmail.com> wrote:
> >> >
> >> >> We could sue Spring-Data but it would introduce a lot of
> dependencies.
> >> >>
> >> >>
> >> >> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
> >> >>
> >> >>> On 13 November 2013 21:59, Philippe Mouawad <
> >> philippe.mouawad@gmail.com>
> >> >>> wrote:
> >> >>> > Hello sebb,
> >> >>> > Shall I commit this development or do we wait for 2.10.1 to be
> >> released
> >> >>> ?
> >> >>>
> >> >>> I am still concerned that this addition is specific to the Redis
> >> server.
> >> >>> I would much prefer to see a generic solution that can use various
> >> >>> different kinds of servers.
> >> >>>
> >> >>> > When do you plan to commit your devs on keytool ?
> >> >>>
> >> >>
> >> >> If I have time I will do it , I plan the following, if you want more
> let
> >> >> me know:
> >> >> - Add keytool path property
> >> >> - Add popup to be clear about error
> >> >> - Add info about how to fix it
> >> >>
> >> >>>
> >> >>> Sorry, have not been able to spend any time on this recently.
> >> >>>
> >> >>> > Thank you
> >> >>> > Regards
> >> >>> > Philippe
> >> >>> >
> >> >>> >
> >> >>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
> >> >>> philippe.mouawad@gmail.com
> >> >>> >> wrote:
> >> >>> >
> >> >>> >> Hello Sebb,
> >> >>> >> My answers below.
> >> >>> >>
> >> >>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> >> >>> >>
> >> >>> >>> On 2 November 2013 08:28, Philippe Mouawad <
> >> >>> philippe.mouawad@gmail.com>
> >> >>> >>> wrote:
> >> >>> >>> > Hello,
> >> >>> >>> > Can I proceed with commit these up coming days ?
> >> >>> >>>
> >> >>> >>> I'm not sure that the discussion was completed.
> >> >>> >>>
> >> >>> >>> As far as I can tell, the proposal only suits some types of
> >> >>> >>> cloud-based test, and relies on 3rd party servers to hold the
> data.
> >> >>> >>>
> >> >>> >>
> >> >>> >> No in my opinion if fits many scenarios:
> >> >>> >> - Cloud based tests, this one seems to me an important one, as
> Cloud
> >> >>> based
> >> >>> >> usage are increasing
> >> >>> >> - Distributed tests, even if it is possible to do it with CSV,
> >> having
> >> >>> the
> >> >>> >> data in a remote server is much easier to manage than having to
> >> >>> >> split/distribute on servers. Even it is true it requires some
> >> skills to
> >> >>> >> manage correctly the Redis server
> >> >>> >> - Continuous integration tests where also having the data in a
> >> >>> centralised
> >> >>> >> , remote servers is easier than managing CSVs. In this case Redis
> >> >>> server
> >> >>> >> plays the same role as a JDBC repository
> >> >>> >> - Compared to a database it should perform better for high load
> >> tests
> >> >>> >> si
>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
On Sat, Nov 30, 2013 at 10:52 AM, sebb <se...@gmail.com> wrote:
> On 27 November 2013 21:21, Philippe Mouawad <ph...@gmail.com>
> wrote:
> > On Tue, Nov 26, 2013 at 1:05 AM, sebb <se...@gmail.com> wrote:
> >
> >> On 23 November 2013 13:48, Philippe Mouawad <philippe.mouawad@gmail.com
> >
> >> wrote:
> >> > Hello sebb,
> >> > What's your decision on this point ?
> >>
> >> I don't make the decisions - they are ideally arrived at by consensus;
> >> failing that, majority.
> >>
> >> > Still having objection to code commit ?
> >>
> >> Yes, because it seems wrong to include code for a specific database
> >> implementation.
> >> We should be implementing generic solutions.
> >>
> >> So would you be ok to go for Spring-data based implementation ?
> > It will impact number of embedded libraries.
>
> There are other possibilities for NoSQL as well, for example
> http://appscale.cs.ucsb.edu/datastores.html - AppScale
>
Commercial no ?
> http://thrift.apache.org/ - Apache Thrift
>
Didn't dig into it yet, but could you point me to a way to do it if you
already did ?
>
> See also
>
> http://www.oracle.com/technetwork/database/nosqldb/overview/nosql-api-497225.html
>
> http://stackoverflow.com/questions/9252034/is-there-an-emerging-standard-api-for-nosql-databases
>
> http://stackoverflow.com/questions/3439878/are-there-any-nosql-standards-emerging
>
> Given that there is no standard like JDBC available (and there may
> never be), what I think needs to happen is to decide where in JMeter
> such databases may be useful (probably not only CSV Dataset) and then
> define a suitable API for this.
> Users can then implement the API in terms of whatever database they
> wish to use (which might include a JDBC implementation).
> This is not something to be rushed into.
>
I don't totally agree with you, I prefer sometimes that we move forward
with "non perfect" thing and improve it in next steps.
Also I am a bit annoyed that time I spent on this development is kind of
thrown away while it is really easy to use and have tested it rather
thoroughly.
>
> In any case, whatever is decided on belongs in a later version of JMeter.
> We first need to get out the version with various fixes to keytool etc.
>
Dev is ready and could have been commited but would have agreed to wait.
>
> >
> >> > Thanks
> >> > Regards
> >> > Philippe
> >> >
> >> >
> >> > On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
> >> > philippe.mouawad@gmail.com> wrote:
> >> >
> >> >> We could sue Spring-Data but it would introduce a lot of
> dependencies.
> >> >>
> >> >>
> >> >> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
> >> >>
> >> >>> On 13 November 2013 21:59, Philippe Mouawad <
> >> philippe.mouawad@gmail.com>
> >> >>> wrote:
> >> >>> > Hello sebb,
> >> >>> > Shall I commit this development or do we wait for 2.10.1 to be
> >> released
> >> >>> ?
> >> >>>
> >> >>> I am still concerned that this addition is specific to the Redis
> >> server.
> >> >>> I would much prefer to see a generic solution that can use various
> >> >>> different kinds of servers.
> >> >>>
> >> >>> > When do you plan to commit your devs on keytool ?
> >> >>>
> >> >>
> >> >> If I have time I will do it , I plan the following, if you want more
> let
> >> >> me know:
> >> >> - Add keytool path property
> >> >> - Add popup to be clear about error
> >> >> - Add info about how to fix it
> >> >>
> >> >>>
> >> >>> Sorry, have not been able to spend any time on this recently.
> >> >>>
> >> >>> > Thank you
> >> >>> > Regards
> >> >>> > Philippe
> >> >>> >
> >> >>> >
> >> >>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
> >> >>> philippe.mouawad@gmail.com
> >> >>> >> wrote:
> >> >>> >
> >> >>> >> Hello Sebb,
> >> >>> >> My answers below.
> >> >>> >>
> >> >>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> >> >>> >>
> >> >>> >>> On 2 November 2013 08:28, Philippe Mouawad <
> >> >>> philippe.mouawad@gmail.com>
> >> >>> >>> wrote:
> >> >>> >>> > Hello,
> >> >>> >>> > Can I proceed with commit these up coming days ?
> >> >>> >>>
> >> >>> >>> I'm not sure that the discussion was completed.
> >> >>> >>>
> >> >>> >>> As far as I can tell, the proposal only suits some types of
> >> >>> >>> cloud-based test, and relies on 3rd party servers to hold the
> data.
> >> >>> >>>
> >> >>> >>
> >> >>> >> No in my opinion if fits many scenarios:
> >> >>> >> - Cloud based tests, this one seems to me an important one, as
> Cloud
> >> >>> based
> >> >>> >> usage are increasing
> >> >>> >> - Distributed tests, even if it is possible to do it with CSV,
> >> having
> >> >>> the
> >> >>> >> data in a remote server is much easier to manage than having to
> >> >>> >> split/distribute on servers. Even it is true it requires some
> >> skills to
> >> >>> >> manage correctly the Redis server
> >> >>> >> - Continuous integration tests where also having the data in a
> >> >>> centralised
> >> >>> >> , remote servers is easier than managing CSVs. In this case Redis
> >> >>> server
> >> >>> >> plays the same role as a JDBC repository
> >> >>> >> - Compared to a database it should perform better for high load
> >> tests
> >> >>> >> since it's an in-memory repo (although this can be done in SQL
> >> >>> databases),
> >> >>> >> see http://redis.io/topics/benchmarks
> >> >>> >>
> >> >>> >>
> >> >>> >>> I'm not yet convinced that this is how we should be extending
> >> JMeter.
> >> >>> >>>
> >> >>> >>
> >> >>> >> I don't understand this argument, it would be another string for
> >> >>> JMeter,
> >> >>> >> my implementation is only 2 classes + properties for I18N ?
> >> >>> >>
> >> >>> >> What do you propose in this case ?
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >>> > Thanks
> >> >>> >>> > Regards
> >> >>> >>> > Philippe
> >> >>> >>> >
> >> >>> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
> >> >>> >>> >
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >> On Monday, October 28, 2013, sebb wrote:
> >> >>> >>> >>
> >> >>> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
> >> >>> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
> >> >>> >>> philippe.mouawad@gmail.com>
> >> >>> >>> >>> wrote:
> >> >>> >>> >>> >> Hello,
> >> >>> >>> >>> >> These days Cloud based testing is becoming popular and
> >> having
> >> >>> to
> >> >>> >>> >>> distribute
> >> >>> >>> >>> >> test data accross many servers through CSV can become
> >> painful
> >> >>> if
> >> >>> >>> not
> >> >>> >>> >>> >> impossible.
> >> >>> >>> >>> >>
> >> >>> >>> >>> >> Even without Cloud, when using distributed testing you
> >> always
> >> >>> have
> >> >>> >>> to
> >> >>> >>> >>> >> replicate your data on all servers, which is a painful
> >> manual
> >> >>> step.
> >> >>> >>> >>> >>
> >> >>> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for
> these
> >> >>> use
> >> >>> >>> cases
> >> >>> >>> >>> ?
> >> >>> >>> >>> >>
> >> >>> >>> >>> >> We could do it in many different ways:
> >> >>> >>> >>> >> - Integrate an automatic CSV replicator, this would
> remain
> >> >>> simple
> >> >>> >>> and
> >> >>> >>> >>> would
> >> >>> >>> >>> >> not introduce any tier, but with cloud based it would not
> >> >>> >>> horizontally
> >> >>> >>> >>> >> scale easily
> >> >>> >>> >>> >
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>> > Not sure I follow the scaling argument. The file would
> only
> >> >>> have to
> >> >>> >>> be
> >> >>> >>> >>> > copied once before the test proper starts.
> >> >>> >>> >>> > From then on, the data is accessed locally.
> >> >>> >>> >>> >
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >> The scale word was not good, In my thinking the issue is more
> >> about
> >> >>> >>> >> deploying/copying/splitting the file among nodes or cloud
> >> members.
> >> >>> A
> >> >>> >>> >> centralized access makes it far easier.
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>> > However, with a database, each node will need at least one
> >> >>> >>> connection
> >> >>> >>> >>> > to the database, and every time more data is needed there
> >> will
> >> >>> be
> >> >>> >>> >>> > network traffic.
> >> >>> >>> >>> > Or the database has to be running on the server node.
> >> >>> >>> >>> >
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >> Agree, I was not saying anything different. But as I said
> this
> >> can
> >> >>> be
> >> >>> >>> >> useful for middle or low volume
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it
> >> performs
> >> >>> >>> fine, and
> >> >>> >>> >>> >> jdbc protocol is not well suited for cloud based
> deployment
> >> >>> (But it
> >> >>> >>> >>> could
> >> >>> >>> >>> >> also be an interesting feature for Continuous
> Integration)
> >> >>> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best
> >> choice)
> >> >>> , see
> >> >>> >>> >>> this
> >> >>> >>> >>> >> high level summary which I find interesting
> >> >>> >>> >>> >>
> >> >>> >>> >>>
> >> >>> >>>
> >> >>>
> >>
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
> >> >>> >>> >>> >>
> >> >>> >>> >>> >> I have implemented a new Redis (based on Java library for
> >> >>> Redis)
> >> >>> >>> >>> DataSet
> >> >>> >>> >>> >> which I plan to commit if no objection.
> >> >>> >>> >>> >>
> >> >>> >>> >>> >> It will introduce 2 new dependencies with Apache License:
> >> >>> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
> >> >>> >>> >>> >> - commons-pool
> >> >>> >>> >>> >
> >> >>> >>> >>> > There is also a dependency on Redis, but I guess that
> would
> >> not
> >> >>> be
> >> >>> >>> >>> bundled.
> >> >>> >>> >>>
> >> >>> >>> >>>
> >> >>> >>> >> no need to anything else than jedis + commons-pool
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>> I've just noticed that Redis is provided as source which
> needs
> >> to
> >> >>> be
> >> >>> >>> >>> built before use.
> >> >>> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems
> >> to me
> >> >>> >>> >>> it's going to be much harder to install Redis.
> >> >>> >>> >>> In which case additional network traffic will be needed to
> >> access
> >> >>> the
> >> >>> >>> >>> database.
> >> >>> >>> >>>
> >> >>> >>> >>> Also there is no official Windows release.
> >> >>> >>> >>>
> >> >>> >>> >>> >> Thoughts ?
> >> >>> >>> >>> >
> >> >>> >>> >>> > Is MongoDB not suitable?
> >> >>> >>> >>> > We already include a jar for it.
> >> >>> >>> >>> >
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >> Mongodb is more document oriented and has another type of
> use
> >> >>> cases.
> >> >>> >>> One
> >> >>> >>> >> interesting feature of redis is lists where you can pop a
> line
> >> it
> >> >>> will
> >> >>> >>> not
> >> >>> >>> >> be available to next calls, it is suitable for tests that
> need
> >> >>> >>> uniqueness
> >> >>> >>> >> accross all nodes.
> >> >>> >>> >>
> >> >>> >>> >>> >> --
> >> >>> >>> >>> >> Regards.
> >> >>> >>> >>> >> Philippe M.
> >> >>> >>> >>> >> @philmdot <https://twitter.com/philmdot>
> >> >>> >>> >>>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >> --
> >> >>> >>> >> Cordialement.
> >> >>> >>> >> Philippe Mouawad.
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >>
> >> >>> >>> >
> >> >>> >>> > --
> >> >>> >>> > Cordialement.
> >> >>> >>> > Philippe Mouawad.
> >> >>> >>>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >> --
> >> >>> >> Cordialement.
> >> >>> >> Philippe Mouawad.
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >
> >> >>> >
> >> >>> > --
> >> >>> > Cordialement.
> >> >>> > Philippe Mouawad.
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Cordialement.
> >> >> Philippe Mouawad.
> >> >>
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Cordialement.
> >> > Philippe Mouawad.
> >>
> >
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by sebb <se...@gmail.com>.
On 27 November 2013 21:21, Philippe Mouawad <ph...@gmail.com> wrote:
> On Tue, Nov 26, 2013 at 1:05 AM, sebb <se...@gmail.com> wrote:
>
>> On 23 November 2013 13:48, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>> > Hello sebb,
>> > What's your decision on this point ?
>>
>> I don't make the decisions - they are ideally arrived at by consensus;
>> failing that, majority.
>>
>> > Still having objection to code commit ?
>>
>> Yes, because it seems wrong to include code for a specific database
>> implementation.
>> We should be implementing generic solutions.
>>
>> So would you be ok to go for Spring-data based implementation ?
> It will impact number of embedded libraries.
There are other possibilities for NoSQL as well, for example
http://appscale.cs.ucsb.edu/datastores.html - AppScale
http://thrift.apache.org/ - Apache Thrift
See also
http://www.oracle.com/technetwork/database/nosqldb/overview/nosql-api-497225.html
http://stackoverflow.com/questions/9252034/is-there-an-emerging-standard-api-for-nosql-databases
http://stackoverflow.com/questions/3439878/are-there-any-nosql-standards-emerging
Given that there is no standard like JDBC available (and there may
never be), what I think needs to happen is to decide where in JMeter
such databases may be useful (probably not only CSV Dataset) and then
define a suitable API for this.
Users can then implement the API in terms of whatever database they
wish to use (which might include a JDBC implementation).
This is not something to be rushed into.
In any case, whatever is decided on belongs in a later version of JMeter.
We first need to get out the version with various fixes to keytool etc.
>
>> > Thanks
>> > Regards
>> > Philippe
>> >
>> >
>> > On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
>> > philippe.mouawad@gmail.com> wrote:
>> >
>> >> We could sue Spring-Data but it would introduce a lot of dependencies.
>> >>
>> >>
>> >> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
>> >>
>> >>> On 13 November 2013 21:59, Philippe Mouawad <
>> philippe.mouawad@gmail.com>
>> >>> wrote:
>> >>> > Hello sebb,
>> >>> > Shall I commit this development or do we wait for 2.10.1 to be
>> released
>> >>> ?
>> >>>
>> >>> I am still concerned that this addition is specific to the Redis
>> server.
>> >>> I would much prefer to see a generic solution that can use various
>> >>> different kinds of servers.
>> >>>
>> >>> > When do you plan to commit your devs on keytool ?
>> >>>
>> >>
>> >> If I have time I will do it , I plan the following, if you want more let
>> >> me know:
>> >> - Add keytool path property
>> >> - Add popup to be clear about error
>> >> - Add info about how to fix it
>> >>
>> >>>
>> >>> Sorry, have not been able to spend any time on this recently.
>> >>>
>> >>> > Thank you
>> >>> > Regards
>> >>> > Philippe
>> >>> >
>> >>> >
>> >>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
>> >>> philippe.mouawad@gmail.com
>> >>> >> wrote:
>> >>> >
>> >>> >> Hello Sebb,
>> >>> >> My answers below.
>> >>> >>
>> >>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
>> >>> >>
>> >>> >>> On 2 November 2013 08:28, Philippe Mouawad <
>> >>> philippe.mouawad@gmail.com>
>> >>> >>> wrote:
>> >>> >>> > Hello,
>> >>> >>> > Can I proceed with commit these up coming days ?
>> >>> >>>
>> >>> >>> I'm not sure that the discussion was completed.
>> >>> >>>
>> >>> >>> As far as I can tell, the proposal only suits some types of
>> >>> >>> cloud-based test, and relies on 3rd party servers to hold the data.
>> >>> >>>
>> >>> >>
>> >>> >> No in my opinion if fits many scenarios:
>> >>> >> - Cloud based tests, this one seems to me an important one, as Cloud
>> >>> based
>> >>> >> usage are increasing
>> >>> >> - Distributed tests, even if it is possible to do it with CSV,
>> having
>> >>> the
>> >>> >> data in a remote server is much easier to manage than having to
>> >>> >> split/distribute on servers. Even it is true it requires some
>> skills to
>> >>> >> manage correctly the Redis server
>> >>> >> - Continuous integration tests where also having the data in a
>> >>> centralised
>> >>> >> , remote servers is easier than managing CSVs. In this case Redis
>> >>> server
>> >>> >> plays the same role as a JDBC repository
>> >>> >> - Compared to a database it should perform better for high load
>> tests
>> >>> >> since it's an in-memory repo (although this can be done in SQL
>> >>> databases),
>> >>> >> see http://redis.io/topics/benchmarks
>> >>> >>
>> >>> >>
>> >>> >>> I'm not yet convinced that this is how we should be extending
>> JMeter.
>> >>> >>>
>> >>> >>
>> >>> >> I don't understand this argument, it would be another string for
>> >>> JMeter,
>> >>> >> my implementation is only 2 classes + properties for I18N ?
>> >>> >>
>> >>> >> What do you propose in this case ?
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>> > Thanks
>> >>> >>> > Regards
>> >>> >>> > Philippe
>> >>> >>> >
>> >>> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> On Monday, October 28, 2013, sebb wrote:
>> >>> >>> >>
>> >>> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>> >>> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
>> >>> >>> philippe.mouawad@gmail.com>
>> >>> >>> >>> wrote:
>> >>> >>> >>> >> Hello,
>> >>> >>> >>> >> These days Cloud based testing is becoming popular and
>> having
>> >>> to
>> >>> >>> >>> distribute
>> >>> >>> >>> >> test data accross many servers through CSV can become
>> painful
>> >>> if
>> >>> >>> not
>> >>> >>> >>> >> impossible.
>> >>> >>> >>> >>
>> >>> >>> >>> >> Even without Cloud, when using distributed testing you
>> always
>> >>> have
>> >>> >>> to
>> >>> >>> >>> >> replicate your data on all servers, which is a painful
>> manual
>> >>> step.
>> >>> >>> >>> >>
>> >>> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for these
>> >>> use
>> >>> >>> cases
>> >>> >>> >>> ?
>> >>> >>> >>> >>
>> >>> >>> >>> >> We could do it in many different ways:
>> >>> >>> >>> >> - Integrate an automatic CSV replicator, this would remain
>> >>> simple
>> >>> >>> and
>> >>> >>> >>> would
>> >>> >>> >>> >> not introduce any tier, but with cloud based it would not
>> >>> >>> horizontally
>> >>> >>> >>> >> scale easily
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> > Not sure I follow the scaling argument. The file would only
>> >>> have to
>> >>> >>> be
>> >>> >>> >>> > copied once before the test proper starts.
>> >>> >>> >>> > From then on, the data is accessed locally.
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> The scale word was not good, In my thinking the issue is more
>> about
>> >>> >>> >> deploying/copying/splitting the file among nodes or cloud
>> members.
>> >>> A
>> >>> >>> >> centralized access makes it far easier.
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> > However, with a database, each node will need at least one
>> >>> >>> connection
>> >>> >>> >>> > to the database, and every time more data is needed there
>> will
>> >>> be
>> >>> >>> >>> > network traffic.
>> >>> >>> >>> > Or the database has to be running on the server node.
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> Agree, I was not saying anything different. But as I said this
>> can
>> >>> be
>> >>> >>> >> useful for middle or low volume
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it
>> performs
>> >>> >>> fine, and
>> >>> >>> >>> >> jdbc protocol is not well suited for cloud based deployment
>> >>> (But it
>> >>> >>> >>> could
>> >>> >>> >>> >> also be an interesting feature for Continuous Integration)
>> >>> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best
>> choice)
>> >>> , see
>> >>> >>> >>> this
>> >>> >>> >>> >> high level summary which I find interesting
>> >>> >>> >>> >>
>> >>> >>> >>>
>> >>> >>>
>> >>>
>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>> >>> >>> >>> >>
>> >>> >>> >>> >> I have implemented a new Redis (based on Java library for
>> >>> Redis)
>> >>> >>> >>> DataSet
>> >>> >>> >>> >> which I plan to commit if no objection.
>> >>> >>> >>> >>
>> >>> >>> >>> >> It will introduce 2 new dependencies with Apache License:
>> >>> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
>> >>> >>> >>> >> - commons-pool
>> >>> >>> >>> >
>> >>> >>> >>> > There is also a dependency on Redis, but I guess that would
>> not
>> >>> be
>> >>> >>> >>> bundled.
>> >>> >>> >>>
>> >>> >>> >>>
>> >>> >>> >> no need to anything else than jedis + commons-pool
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> I've just noticed that Redis is provided as source which needs
>> to
>> >>> be
>> >>> >>> >>> built before use.
>> >>> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems
>> to me
>> >>> >>> >>> it's going to be much harder to install Redis.
>> >>> >>> >>> In which case additional network traffic will be needed to
>> access
>> >>> the
>> >>> >>> >>> database.
>> >>> >>> >>>
>> >>> >>> >>> Also there is no official Windows release.
>> >>> >>> >>>
>> >>> >>> >>> >> Thoughts ?
>> >>> >>> >>> >
>> >>> >>> >>> > Is MongoDB not suitable?
>> >>> >>> >>> > We already include a jar for it.
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> Mongodb is more document oriented and has another type of use
>> >>> cases.
>> >>> >>> One
>> >>> >>> >> interesting feature of redis is lists where you can pop a line
>> it
>> >>> will
>> >>> >>> not
>> >>> >>> >> be available to next calls, it is suitable for tests that need
>> >>> >>> uniqueness
>> >>> >>> >> accross all nodes.
>> >>> >>> >>
>> >>> >>> >>> >> --
>> >>> >>> >>> >> Regards.
>> >>> >>> >>> >> Philippe M.
>> >>> >>> >>> >> @philmdot <https://twitter.com/philmdot>
>> >>> >>> >>>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> --
>> >>> >>> >> Cordialement.
>> >>> >>> >> Philippe Mouawad.
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >
>> >>> >>> > --
>> >>> >>> > Cordialement.
>> >>> >>> > Philippe Mouawad.
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Cordialement.
>> >>> >> Philippe Mouawad.
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Cordialement.
>> >>> > Philippe Mouawad.
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Cordialement.
>> >> Philippe Mouawad.
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
Any feedback?
Thanks
On Wednesday, November 27, 2013, Philippe Mouawad wrote:
>
>
>
> On Tue, Nov 26, 2013 at 1:05 AM, sebb <sebbaz@gmail.com<javascript:_e({}, 'cvml', 'sebbaz@gmail.com');>
> > wrote:
>
>> On 23 November 2013 13:48, Philippe Mouawad <philippe.mouawad@gmail.com<javascript:_e({}, 'cvml', 'philippe.mouawad@gmail.com');>>
>> wrote:
>> > Hello sebb,
>> > What's your decision on this point ?
>>
>> I don't make the decisions - they are ideally arrived at by consensus;
>> failing that, majority.
>>
>> > Still having objection to code commit ?
>>
>> Yes, because it seems wrong to include code for a specific database
>> implementation.
>> We should be implementing generic solutions.
>>
>> So would you be ok to go for Spring-data based implementation ?
> It will impact number of embedded libraries.
>
>
> > Thanks
> > Regards
> > Philippe
> >
> >
> > On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
> > philippe.mouawad@gmail.com> wrote:
> >
> >> We could sue Spring-Data but it would introduce a lot of dependencies.
> >>
> >>
> >> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
> >>
> >>> On 13 November 2013 21:59, Philippe Mouawad <
> philippe.mouawad@gmail.com>
> >>> wrote:
> >>> > Hello sebb,
> >>> > Shall I commit this development or do we wait for 2.10.1 to be
> released
> >>> ?
> >>>
> >>> I am still concerned that this addition is specific to the Redis
> server.
> >>> I would much prefer to see a generic solution that can use various
> >>> different kinds of servers.
> >>>
> >>> > When do you plan to commit your devs on keytool ?
> >>>
> >>
> >> If I have time I will do it , I plan the following, if you want more let
> >> me know:
> >> - Add keytool path property
> >> - Add popup to be clear about error
> >> - Add info about how to fix it
> >>
> >>>
> >>> Sorry, have not been able to spend any time on this recently.
> >>>
> >>> > Thank you
> >>> > Regards
> >>> > Philippe
> >>> >
> >>> >
> >>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
> >>> philippe.mouawad@gmail.com
> >>> >> wrote:
> >>> >
> >>> >> Hello Sebb,
> >>> >> My answers below.
> >>> >>
> >>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> >>> >>
> >>> >>> On 2 November 2013 08:28, Philippe Mouawad <
> >>> philippe.mouawad@gmail.com>
> >>> >>> wrote:
> >>> >>> > Hello,
> >>> >>> > Can I proceed with commit these up coming days ?
> >>> >>>
> >>> >>> I'm not sure that the discussion was completed.
> >>> >>>
> >>> >>> As far as I can tell, the proposal only suits some types of
> >>> >>> cloud-based test, and relies on 3rd party servers to hold the data.
> >>> >>>
> >>> >>
> >>> >> No in my opinion if fits many scenarios:
> >>> >> - Cloud based tests, this one seems to me an important one, as Cloud
> >>> based
> >>> >> usage are increasing
> >>> >> - Distributed tests, even if it is possible to do it with CSV,
> having
> >>> the
> >>> >> data in a remote server is much easier to manage than having to
> >>> >> split/distribute on servers. Even it is true it requires some
> skills to
> >>> >> manage correctly the Redis server
> >>> >> - Continuous integration tests where also having the data in a
> >>> centralised
> >>> >> , remote servers is easier than managing CSVs. In this case Redis
> >>> server
> >>> >> plays the same role as a JDBC repository
> >>> >> - Compared to a database it should perform better for high load
> tests
> >>> >> since it's an in-memory repo (although this can be done in SQL
> >>> databases),
> >>> >> see http://redis.io/topics/benchmarks
> >>> >>
> >>> >>
> >>> >>> I'm not yet convinced that this is how we should be extending
> JMeter.
> >>> >>>
> >>> >>
> >>> >> I don't understand this argument, it would be another string for
> >>> JMeter,
> >>> >> my implementation is only 2 classes + properties for I18N ?
> >>> >>
> >>> >> What do you propose in this case ?
> >>> >>
> >>
>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
On Tue, Nov 26, 2013 at 1:05 AM, sebb <se...@gmail.com> wrote:
> On 23 November 2013 13:48, Philippe Mouawad <ph...@gmail.com>
> wrote:
> > Hello sebb,
> > What's your decision on this point ?
>
> I don't make the decisions - they are ideally arrived at by consensus;
> failing that, majority.
>
> > Still having objection to code commit ?
>
> Yes, because it seems wrong to include code for a specific database
> implementation.
> We should be implementing generic solutions.
>
> So would you be ok to go for Spring-data based implementation ?
It will impact number of embedded libraries.
> > Thanks
> > Regards
> > Philippe
> >
> >
> > On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
> > philippe.mouawad@gmail.com> wrote:
> >
> >> We could sue Spring-Data but it would introduce a lot of dependencies.
> >>
> >>
> >> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
> >>
> >>> On 13 November 2013 21:59, Philippe Mouawad <
> philippe.mouawad@gmail.com>
> >>> wrote:
> >>> > Hello sebb,
> >>> > Shall I commit this development or do we wait for 2.10.1 to be
> released
> >>> ?
> >>>
> >>> I am still concerned that this addition is specific to the Redis
> server.
> >>> I would much prefer to see a generic solution that can use various
> >>> different kinds of servers.
> >>>
> >>> > When do you plan to commit your devs on keytool ?
> >>>
> >>
> >> If I have time I will do it , I plan the following, if you want more let
> >> me know:
> >> - Add keytool path property
> >> - Add popup to be clear about error
> >> - Add info about how to fix it
> >>
> >>>
> >>> Sorry, have not been able to spend any time on this recently.
> >>>
> >>> > Thank you
> >>> > Regards
> >>> > Philippe
> >>> >
> >>> >
> >>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
> >>> philippe.mouawad@gmail.com
> >>> >> wrote:
> >>> >
> >>> >> Hello Sebb,
> >>> >> My answers below.
> >>> >>
> >>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> >>> >>
> >>> >>> On 2 November 2013 08:28, Philippe Mouawad <
> >>> philippe.mouawad@gmail.com>
> >>> >>> wrote:
> >>> >>> > Hello,
> >>> >>> > Can I proceed with commit these up coming days ?
> >>> >>>
> >>> >>> I'm not sure that the discussion was completed.
> >>> >>>
> >>> >>> As far as I can tell, the proposal only suits some types of
> >>> >>> cloud-based test, and relies on 3rd party servers to hold the data.
> >>> >>>
> >>> >>
> >>> >> No in my opinion if fits many scenarios:
> >>> >> - Cloud based tests, this one seems to me an important one, as Cloud
> >>> based
> >>> >> usage are increasing
> >>> >> - Distributed tests, even if it is possible to do it with CSV,
> having
> >>> the
> >>> >> data in a remote server is much easier to manage than having to
> >>> >> split/distribute on servers. Even it is true it requires some
> skills to
> >>> >> manage correctly the Redis server
> >>> >> - Continuous integration tests where also having the data in a
> >>> centralised
> >>> >> , remote servers is easier than managing CSVs. In this case Redis
> >>> server
> >>> >> plays the same role as a JDBC repository
> >>> >> - Compared to a database it should perform better for high load
> tests
> >>> >> since it's an in-memory repo (although this can be done in SQL
> >>> databases),
> >>> >> see http://redis.io/topics/benchmarks
> >>> >>
> >>> >>
> >>> >>> I'm not yet convinced that this is how we should be extending
> JMeter.
> >>> >>>
> >>> >>
> >>> >> I don't understand this argument, it would be another string for
> >>> JMeter,
> >>> >> my implementation is only 2 classes + properties for I18N ?
> >>> >>
> >>> >> What do you propose in this case ?
> >>> >>
> >>> >>
> >>> >>
> >>> >>> > Thanks
> >>> >>> > Regards
> >>> >>> > Philippe
> >>> >>> >
> >>> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
> >>> >>> >
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> On Monday, October 28, 2013, sebb wrote:
> >>> >>> >>
> >>> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
> >>> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
> >>> >>> philippe.mouawad@gmail.com>
> >>> >>> >>> wrote:
> >>> >>> >>> >> Hello,
> >>> >>> >>> >> These days Cloud based testing is becoming popular and
> having
> >>> to
> >>> >>> >>> distribute
> >>> >>> >>> >> test data accross many servers through CSV can become
> painful
> >>> if
> >>> >>> not
> >>> >>> >>> >> impossible.
> >>> >>> >>> >>
> >>> >>> >>> >> Even without Cloud, when using distributed testing you
> always
> >>> have
> >>> >>> to
> >>> >>> >>> >> replicate your data on all servers, which is a painful
> manual
> >>> step.
> >>> >>> >>> >>
> >>> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for these
> >>> use
> >>> >>> cases
> >>> >>> >>> ?
> >>> >>> >>> >>
> >>> >>> >>> >> We could do it in many different ways:
> >>> >>> >>> >> - Integrate an automatic CSV replicator, this would remain
> >>> simple
> >>> >>> and
> >>> >>> >>> would
> >>> >>> >>> >> not introduce any tier, but with cloud based it would not
> >>> >>> horizontally
> >>> >>> >>> >> scale easily
> >>> >>> >>> >
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>> > Not sure I follow the scaling argument. The file would only
> >>> have to
> >>> >>> be
> >>> >>> >>> > copied once before the test proper starts.
> >>> >>> >>> > From then on, the data is accessed locally.
> >>> >>> >>> >
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> The scale word was not good, In my thinking the issue is more
> about
> >>> >>> >> deploying/copying/splitting the file among nodes or cloud
> members.
> >>> A
> >>> >>> >> centralized access makes it far easier.
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>> > However, with a database, each node will need at least one
> >>> >>> connection
> >>> >>> >>> > to the database, and every time more data is needed there
> will
> >>> be
> >>> >>> >>> > network traffic.
> >>> >>> >>> > Or the database has to be running on the server node.
> >>> >>> >>> >
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> Agree, I was not saying anything different. But as I said this
> can
> >>> be
> >>> >>> >> useful for middle or low volume
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it
> performs
> >>> >>> fine, and
> >>> >>> >>> >> jdbc protocol is not well suited for cloud based deployment
> >>> (But it
> >>> >>> >>> could
> >>> >>> >>> >> also be an interesting feature for Continuous Integration)
> >>> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best
> choice)
> >>> , see
> >>> >>> >>> this
> >>> >>> >>> >> high level summary which I find interesting
> >>> >>> >>> >>
> >>> >>> >>>
> >>> >>>
> >>>
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
> >>> >>> >>> >>
> >>> >>> >>> >> I have implemented a new Redis (based on Java library for
> >>> Redis)
> >>> >>> >>> DataSet
> >>> >>> >>> >> which I plan to commit if no objection.
> >>> >>> >>> >>
> >>> >>> >>> >> It will introduce 2 new dependencies with Apache License:
> >>> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
> >>> >>> >>> >> - commons-pool
> >>> >>> >>> >
> >>> >>> >>> > There is also a dependency on Redis, but I guess that would
> not
> >>> be
> >>> >>> >>> bundled.
> >>> >>> >>>
> >>> >>> >>>
> >>> >>> >> no need to anything else than jedis + commons-pool
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>> I've just noticed that Redis is provided as source which needs
> to
> >>> be
> >>> >>> >>> built before use.
> >>> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems
> to me
> >>> >>> >>> it's going to be much harder to install Redis.
> >>> >>> >>> In which case additional network traffic will be needed to
> access
> >>> the
> >>> >>> >>> database.
> >>> >>> >>>
> >>> >>> >>> Also there is no official Windows release.
> >>> >>> >>>
> >>> >>> >>> >> Thoughts ?
> >>> >>> >>> >
> >>> >>> >>> > Is MongoDB not suitable?
> >>> >>> >>> > We already include a jar for it.
> >>> >>> >>> >
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> Mongodb is more document oriented and has another type of use
> >>> cases.
> >>> >>> One
> >>> >>> >> interesting feature of redis is lists where you can pop a line
> it
> >>> will
> >>> >>> not
> >>> >>> >> be available to next calls, it is suitable for tests that need
> >>> >>> uniqueness
> >>> >>> >> accross all nodes.
> >>> >>> >>
> >>> >>> >>> >> --
> >>> >>> >>> >> Regards.
> >>> >>> >>> >> Philippe M.
> >>> >>> >>> >> @philmdot <https://twitter.com/philmdot>
> >>> >>> >>>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> --
> >>> >>> >> Cordialement.
> >>> >>> >> Philippe Mouawad.
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >>
> >>> >>> >
> >>> >>> > --
> >>> >>> > Cordialement.
> >>> >>> > Philippe Mouawad.
> >>> >>>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Cordialement.
> >>> >> Philippe Mouawad.
> >>> >>
> >>> >>
> >>> >>
> >>> >
> >>> >
> >>> > --
> >>> > Cordialement.
> >>> > Philippe Mouawad.
> >>>
> >>
> >>
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >>
> >>
> >>
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by sebb <se...@gmail.com>.
On 23 November 2013 13:48, Philippe Mouawad <ph...@gmail.com> wrote:
> Hello sebb,
> What's your decision on this point ?
I don't make the decisions - they are ideally arrived at by consensus;
failing that, majority.
> Still having objection to code commit ?
Yes, because it seems wrong to include code for a specific database
implementation.
We should be implementing generic solutions.
> Thanks
> Regards
> Philippe
>
>
> On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
> philippe.mouawad@gmail.com> wrote:
>
>> We could sue Spring-Data but it would introduce a lot of dependencies.
>>
>>
>> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
>>
>>> On 13 November 2013 21:59, Philippe Mouawad <ph...@gmail.com>
>>> wrote:
>>> > Hello sebb,
>>> > Shall I commit this development or do we wait for 2.10.1 to be released
>>> ?
>>>
>>> I am still concerned that this addition is specific to the Redis server.
>>> I would much prefer to see a generic solution that can use various
>>> different kinds of servers.
>>>
>>> > When do you plan to commit your devs on keytool ?
>>>
>>
>> If I have time I will do it , I plan the following, if you want more let
>> me know:
>> - Add keytool path property
>> - Add popup to be clear about error
>> - Add info about how to fix it
>>
>>>
>>> Sorry, have not been able to spend any time on this recently.
>>>
>>> > Thank you
>>> > Regards
>>> > Philippe
>>> >
>>> >
>>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
>>> philippe.mouawad@gmail.com
>>> >> wrote:
>>> >
>>> >> Hello Sebb,
>>> >> My answers below.
>>> >>
>>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
>>> >>
>>> >>> On 2 November 2013 08:28, Philippe Mouawad <
>>> philippe.mouawad@gmail.com>
>>> >>> wrote:
>>> >>> > Hello,
>>> >>> > Can I proceed with commit these up coming days ?
>>> >>>
>>> >>> I'm not sure that the discussion was completed.
>>> >>>
>>> >>> As far as I can tell, the proposal only suits some types of
>>> >>> cloud-based test, and relies on 3rd party servers to hold the data.
>>> >>>
>>> >>
>>> >> No in my opinion if fits many scenarios:
>>> >> - Cloud based tests, this one seems to me an important one, as Cloud
>>> based
>>> >> usage are increasing
>>> >> - Distributed tests, even if it is possible to do it with CSV, having
>>> the
>>> >> data in a remote server is much easier to manage than having to
>>> >> split/distribute on servers. Even it is true it requires some skills to
>>> >> manage correctly the Redis server
>>> >> - Continuous integration tests where also having the data in a
>>> centralised
>>> >> , remote servers is easier than managing CSVs. In this case Redis
>>> server
>>> >> plays the same role as a JDBC repository
>>> >> - Compared to a database it should perform better for high load tests
>>> >> since it's an in-memory repo (although this can be done in SQL
>>> databases),
>>> >> see http://redis.io/topics/benchmarks
>>> >>
>>> >>
>>> >>> I'm not yet convinced that this is how we should be extending JMeter.
>>> >>>
>>> >>
>>> >> I don't understand this argument, it would be another string for
>>> JMeter,
>>> >> my implementation is only 2 classes + properties for I18N ?
>>> >>
>>> >> What do you propose in this case ?
>>> >>
>>> >>
>>> >>
>>> >>> > Thanks
>>> >>> > Regards
>>> >>> > Philippe
>>> >>> >
>>> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
>>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >> On Monday, October 28, 2013, sebb wrote:
>>> >>> >>
>>> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>>> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
>>> >>> philippe.mouawad@gmail.com>
>>> >>> >>> wrote:
>>> >>> >>> >> Hello,
>>> >>> >>> >> These days Cloud based testing is becoming popular and having
>>> to
>>> >>> >>> distribute
>>> >>> >>> >> test data accross many servers through CSV can become painful
>>> if
>>> >>> not
>>> >>> >>> >> impossible.
>>> >>> >>> >>
>>> >>> >>> >> Even without Cloud, when using distributed testing you always
>>> have
>>> >>> to
>>> >>> >>> >> replicate your data on all servers, which is a painful manual
>>> step.
>>> >>> >>> >>
>>> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for these
>>> use
>>> >>> cases
>>> >>> >>> ?
>>> >>> >>> >>
>>> >>> >>> >> We could do it in many different ways:
>>> >>> >>> >> - Integrate an automatic CSV replicator, this would remain
>>> simple
>>> >>> and
>>> >>> >>> would
>>> >>> >>> >> not introduce any tier, but with cloud based it would not
>>> >>> horizontally
>>> >>> >>> >> scale easily
>>> >>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >>> > Not sure I follow the scaling argument. The file would only
>>> have to
>>> >>> be
>>> >>> >>> > copied once before the test proper starts.
>>> >>> >>> > From then on, the data is accessed locally.
>>> >>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >> The scale word was not good, In my thinking the issue is more about
>>> >>> >> deploying/copying/splitting the file among nodes or cloud members.
>>> A
>>> >>> >> centralized access makes it far easier.
>>> >>> >>
>>> >>> >>
>>> >>> >>> > However, with a database, each node will need at least one
>>> >>> connection
>>> >>> >>> > to the database, and every time more data is needed there will
>>> be
>>> >>> >>> > network traffic.
>>> >>> >>> > Or the database has to be running on the server node.
>>> >>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >> Agree, I was not saying anything different. But as I said this can
>>> be
>>> >>> >> useful for middle or low volume
>>> >>> >>
>>> >>> >>
>>> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs
>>> >>> fine, and
>>> >>> >>> >> jdbc protocol is not well suited for cloud based deployment
>>> (But it
>>> >>> >>> could
>>> >>> >>> >> also be an interesting feature for Continuous Integration)
>>> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best choice)
>>> , see
>>> >>> >>> this
>>> >>> >>> >> high level summary which I find interesting
>>> >>> >>> >>
>>> >>> >>>
>>> >>>
>>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>>> >>> >>> >>
>>> >>> >>> >> I have implemented a new Redis (based on Java library for
>>> Redis)
>>> >>> >>> DataSet
>>> >>> >>> >> which I plan to commit if no objection.
>>> >>> >>> >>
>>> >>> >>> >> It will introduce 2 new dependencies with Apache License:
>>> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
>>> >>> >>> >> - commons-pool
>>> >>> >>> >
>>> >>> >>> > There is also a dependency on Redis, but I guess that would not
>>> be
>>> >>> >>> bundled.
>>> >>> >>>
>>> >>> >>>
>>> >>> >> no need to anything else than jedis + commons-pool
>>> >>> >>
>>> >>> >>
>>> >>> >>> I've just noticed that Redis is provided as source which needs to
>>> be
>>> >>> >>> built before use.
>>> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>>> >>> >>> it's going to be much harder to install Redis.
>>> >>> >>> In which case additional network traffic will be needed to access
>>> the
>>> >>> >>> database.
>>> >>> >>>
>>> >>> >>> Also there is no official Windows release.
>>> >>> >>>
>>> >>> >>> >> Thoughts ?
>>> >>> >>> >
>>> >>> >>> > Is MongoDB not suitable?
>>> >>> >>> > We already include a jar for it.
>>> >>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >> Mongodb is more document oriented and has another type of use
>>> cases.
>>> >>> One
>>> >>> >> interesting feature of redis is lists where you can pop a line it
>>> will
>>> >>> not
>>> >>> >> be available to next calls, it is suitable for tests that need
>>> >>> uniqueness
>>> >>> >> accross all nodes.
>>> >>> >>
>>> >>> >>> >> --
>>> >>> >>> >> Regards.
>>> >>> >>> >> Philippe M.
>>> >>> >>> >> @philmdot <https://twitter.com/philmdot>
>>> >>> >>>
>>> >>> >>
>>> >>> >>
>>> >>> >> --
>>> >>> >> Cordialement.
>>> >>> >> Philippe Mouawad.
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >
>>> >>> > --
>>> >>> > Cordialement.
>>> >>> > Philippe Mouawad.
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Cordialement.
>>> >> Philippe Mouawad.
>>> >>
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> > Cordialement.
>>> > Philippe Mouawad.
>>>
>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello sebb,
What's your decision on this point ?
Still having objection to code commit ?
Thanks
Regards
Philippe
On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:
> We could sue Spring-Data but it would introduce a lot of dependencies.
>
>
> On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
>
>> On 13 November 2013 21:59, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>> > Hello sebb,
>> > Shall I commit this development or do we wait for 2.10.1 to be released
>> ?
>>
>> I am still concerned that this addition is specific to the Redis server.
>> I would much prefer to see a generic solution that can use various
>> different kinds of servers.
>>
>> > When do you plan to commit your devs on keytool ?
>>
>
> If I have time I will do it , I plan the following, if you want more let
> me know:
> - Add keytool path property
> - Add popup to be clear about error
> - Add info about how to fix it
>
>>
>> Sorry, have not been able to spend any time on this recently.
>>
>> > Thank you
>> > Regards
>> > Philippe
>> >
>> >
>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
>> philippe.mouawad@gmail.com
>> >> wrote:
>> >
>> >> Hello Sebb,
>> >> My answers below.
>> >>
>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
>> >>
>> >>> On 2 November 2013 08:28, Philippe Mouawad <
>> philippe.mouawad@gmail.com>
>> >>> wrote:
>> >>> > Hello,
>> >>> > Can I proceed with commit these up coming days ?
>> >>>
>> >>> I'm not sure that the discussion was completed.
>> >>>
>> >>> As far as I can tell, the proposal only suits some types of
>> >>> cloud-based test, and relies on 3rd party servers to hold the data.
>> >>>
>> >>
>> >> No in my opinion if fits many scenarios:
>> >> - Cloud based tests, this one seems to me an important one, as Cloud
>> based
>> >> usage are increasing
>> >> - Distributed tests, even if it is possible to do it with CSV, having
>> the
>> >> data in a remote server is much easier to manage than having to
>> >> split/distribute on servers. Even it is true it requires some skills to
>> >> manage correctly the Redis server
>> >> - Continuous integration tests where also having the data in a
>> centralised
>> >> , remote servers is easier than managing CSVs. In this case Redis
>> server
>> >> plays the same role as a JDBC repository
>> >> - Compared to a database it should perform better for high load tests
>> >> since it's an in-memory repo (although this can be done in SQL
>> databases),
>> >> see http://redis.io/topics/benchmarks
>> >>
>> >>
>> >>> I'm not yet convinced that this is how we should be extending JMeter.
>> >>>
>> >>
>> >> I don't understand this argument, it would be another string for
>> JMeter,
>> >> my implementation is only 2 classes + properties for I18N ?
>> >>
>> >> What do you propose in this case ?
>> >>
>> >>
>> >>
>> >>> > Thanks
>> >>> > Regards
>> >>> > Philippe
>> >>> >
>> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
>> >>> >
>> >>> >>
>> >>> >>
>> >>> >> On Monday, October 28, 2013, sebb wrote:
>> >>> >>
>> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
>> >>> philippe.mouawad@gmail.com>
>> >>> >>> wrote:
>> >>> >>> >> Hello,
>> >>> >>> >> These days Cloud based testing is becoming popular and having
>> to
>> >>> >>> distribute
>> >>> >>> >> test data accross many servers through CSV can become painful
>> if
>> >>> not
>> >>> >>> >> impossible.
>> >>> >>> >>
>> >>> >>> >> Even without Cloud, when using distributed testing you always
>> have
>> >>> to
>> >>> >>> >> replicate your data on all servers, which is a painful manual
>> step.
>> >>> >>> >>
>> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for these
>> use
>> >>> cases
>> >>> >>> ?
>> >>> >>> >>
>> >>> >>> >> We could do it in many different ways:
>> >>> >>> >> - Integrate an automatic CSV replicator, this would remain
>> simple
>> >>> and
>> >>> >>> would
>> >>> >>> >> not introduce any tier, but with cloud based it would not
>> >>> horizontally
>> >>> >>> >> scale easily
>> >>> >>> >
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>> > Not sure I follow the scaling argument. The file would only
>> have to
>> >>> be
>> >>> >>> > copied once before the test proper starts.
>> >>> >>> > From then on, the data is accessed locally.
>> >>> >>> >
>> >>> >>
>> >>> >>
>> >>> >> The scale word was not good, In my thinking the issue is more about
>> >>> >> deploying/copying/splitting the file among nodes or cloud members.
>> A
>> >>> >> centralized access makes it far easier.
>> >>> >>
>> >>> >>
>> >>> >>> > However, with a database, each node will need at least one
>> >>> connection
>> >>> >>> > to the database, and every time more data is needed there will
>> be
>> >>> >>> > network traffic.
>> >>> >>> > Or the database has to be running on the server node.
>> >>> >>> >
>> >>> >>
>> >>> >>
>> >>> >> Agree, I was not saying anything different. But as I said this can
>> be
>> >>> >> useful for middle or low volume
>> >>> >>
>> >>> >>
>> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs
>> >>> fine, and
>> >>> >>> >> jdbc protocol is not well suited for cloud based deployment
>> (But it
>> >>> >>> could
>> >>> >>> >> also be an interesting feature for Continuous Integration)
>> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best choice)
>> , see
>> >>> >>> this
>> >>> >>> >> high level summary which I find interesting
>> >>> >>> >>
>> >>> >>>
>> >>>
>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>> >>> >>> >>
>> >>> >>> >> I have implemented a new Redis (based on Java library for
>> Redis)
>> >>> >>> DataSet
>> >>> >>> >> which I plan to commit if no objection.
>> >>> >>> >>
>> >>> >>> >> It will introduce 2 new dependencies with Apache License:
>> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
>> >>> >>> >> - commons-pool
>> >>> >>> >
>> >>> >>> > There is also a dependency on Redis, but I guess that would not
>> be
>> >>> >>> bundled.
>> >>> >>>
>> >>> >>>
>> >>> >> no need to anything else than jedis + commons-pool
>> >>> >>
>> >>> >>
>> >>> >>> I've just noticed that Redis is provided as source which needs to
>> be
>> >>> >>> built before use.
>> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>> >>> >>> it's going to be much harder to install Redis.
>> >>> >>> In which case additional network traffic will be needed to access
>> the
>> >>> >>> database.
>> >>> >>>
>> >>> >>> Also there is no official Windows release.
>> >>> >>>
>> >>> >>> >> Thoughts ?
>> >>> >>> >
>> >>> >>> > Is MongoDB not suitable?
>> >>> >>> > We already include a jar for it.
>> >>> >>> >
>> >>> >>
>> >>> >>
>> >>> >> Mongodb is more document oriented and has another type of use
>> cases.
>> >>> One
>> >>> >> interesting feature of redis is lists where you can pop a line it
>> will
>> >>> not
>> >>> >> be available to next calls, it is suitable for tests that need
>> >>> uniqueness
>> >>> >> accross all nodes.
>> >>> >>
>> >>> >>> >> --
>> >>> >>> >> Regards.
>> >>> >>> >> Philippe M.
>> >>> >>> >> @philmdot <https://twitter.com/philmdot>
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Cordialement.
>> >>> >> Philippe Mouawad.
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >
>> >>> > --
>> >>> > Cordialement.
>> >>> > Philippe Mouawad.
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Cordialement.
>> >> Philippe Mouawad.
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
We could sue Spring-Data but it would introduce a lot of dependencies.
On Fri, Nov 15, 2013 at 12:54 AM, sebb <se...@gmail.com> wrote:
> On 13 November 2013 21:59, Philippe Mouawad <ph...@gmail.com>
> wrote:
> > Hello sebb,
> > Shall I commit this development or do we wait for 2.10.1 to be released ?
>
> I am still concerned that this addition is specific to the Redis server.
> I would much prefer to see a generic solution that can use various
> different kinds of servers.
>
> > When do you plan to commit your devs on keytool ?
>
If I have time I will do it , I plan the following, if you want more let me
know:
- Add keytool path property
- Add popup to be clear about error
- Add info about how to fix it
>
> Sorry, have not been able to spend any time on this recently.
>
> > Thank you
> > Regards
> > Philippe
> >
> >
> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
> philippe.mouawad@gmail.com
> >> wrote:
> >
> >> Hello Sebb,
> >> My answers below.
> >>
> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> >>
> >>> On 2 November 2013 08:28, Philippe Mouawad <philippe.mouawad@gmail.com
> >
> >>> wrote:
> >>> > Hello,
> >>> > Can I proceed with commit these up coming days ?
> >>>
> >>> I'm not sure that the discussion was completed.
> >>>
> >>> As far as I can tell, the proposal only suits some types of
> >>> cloud-based test, and relies on 3rd party servers to hold the data.
> >>>
> >>
> >> No in my opinion if fits many scenarios:
> >> - Cloud based tests, this one seems to me an important one, as Cloud
> based
> >> usage are increasing
> >> - Distributed tests, even if it is possible to do it with CSV, having
> the
> >> data in a remote server is much easier to manage than having to
> >> split/distribute on servers. Even it is true it requires some skills to
> >> manage correctly the Redis server
> >> - Continuous integration tests where also having the data in a
> centralised
> >> , remote servers is easier than managing CSVs. In this case Redis server
> >> plays the same role as a JDBC repository
> >> - Compared to a database it should perform better for high load tests
> >> since it's an in-memory repo (although this can be done in SQL
> databases),
> >> see http://redis.io/topics/benchmarks
> >>
> >>
> >>> I'm not yet convinced that this is how we should be extending JMeter.
> >>>
> >>
> >> I don't understand this argument, it would be another string for JMeter,
> >> my implementation is only 2 classes + properties for I18N ?
> >>
> >> What do you propose in this case ?
> >>
> >>
> >>
> >>> > Thanks
> >>> > Regards
> >>> > Philippe
> >>> >
> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
> >>> >
> >>> >>
> >>> >>
> >>> >> On Monday, October 28, 2013, sebb wrote:
> >>> >>
> >>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
> >>> philippe.mouawad@gmail.com>
> >>> >>> wrote:
> >>> >>> >> Hello,
> >>> >>> >> These days Cloud based testing is becoming popular and having to
> >>> >>> distribute
> >>> >>> >> test data accross many servers through CSV can become painful if
> >>> not
> >>> >>> >> impossible.
> >>> >>> >>
> >>> >>> >> Even without Cloud, when using distributed testing you always
> have
> >>> to
> >>> >>> >> replicate your data on all servers, which is a painful manual
> step.
> >>> >>> >>
> >>> >>> >> Shouldn't we introduce a new DataSet more suitable for these use
> >>> cases
> >>> >>> ?
> >>> >>> >>
> >>> >>> >> We could do it in many different ways:
> >>> >>> >> - Integrate an automatic CSV replicator, this would remain
> simple
> >>> and
> >>> >>> would
> >>> >>> >> not introduce any tier, but with cloud based it would not
> >>> horizontally
> >>> >>> >> scale easily
> >>> >>> >
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>> > Not sure I follow the scaling argument. The file would only have
> to
> >>> be
> >>> >>> > copied once before the test proper starts.
> >>> >>> > From then on, the data is accessed locally.
> >>> >>> >
> >>> >>
> >>> >>
> >>> >> The scale word was not good, In my thinking the issue is more about
> >>> >> deploying/copying/splitting the file among nodes or cloud members. A
> >>> >> centralized access makes it far easier.
> >>> >>
> >>> >>
> >>> >>> > However, with a database, each node will need at least one
> >>> connection
> >>> >>> > to the database, and every time more data is needed there will be
> >>> >>> > network traffic.
> >>> >>> > Or the database has to be running on the server node.
> >>> >>> >
> >>> >>
> >>> >>
> >>> >> Agree, I was not saying anything different. But as I said this can
> be
> >>> >> useful for middle or low volume
> >>> >>
> >>> >>
> >>> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs
> >>> fine, and
> >>> >>> >> jdbc protocol is not well suited for cloud based deployment
> (But it
> >>> >>> could
> >>> >>> >> also be an interesting feature for Continuous Integration)
> >>> >>> >> - Use a NOSQL repository (Redis seems to me the best choice) ,
> see
> >>> >>> this
> >>> >>> >> high level summary which I find interesting
> >>> >>> >>
> >>> >>>
> >>>
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
> >>> >>> >>
> >>> >>> >> I have implemented a new Redis (based on Java library for Redis)
> >>> >>> DataSet
> >>> >>> >> which I plan to commit if no objection.
> >>> >>> >>
> >>> >>> >> It will introduce 2 new dependencies with Apache License:
> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
> >>> >>> >> - commons-pool
> >>> >>> >
> >>> >>> > There is also a dependency on Redis, but I guess that would not
> be
> >>> >>> bundled.
> >>> >>>
> >>> >>>
> >>> >> no need to anything else than jedis + commons-pool
> >>> >>
> >>> >>
> >>> >>> I've just noticed that Redis is provided as source which needs to
> be
> >>> >>> built before use.
> >>> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
> >>> >>> it's going to be much harder to install Redis.
> >>> >>> In which case additional network traffic will be needed to access
> the
> >>> >>> database.
> >>> >>>
> >>> >>> Also there is no official Windows release.
> >>> >>>
> >>> >>> >> Thoughts ?
> >>> >>> >
> >>> >>> > Is MongoDB not suitable?
> >>> >>> > We already include a jar for it.
> >>> >>> >
> >>> >>
> >>> >>
> >>> >> Mongodb is more document oriented and has another type of use
> cases.
> >>> One
> >>> >> interesting feature of redis is lists where you can pop a line it
> will
> >>> not
> >>> >> be available to next calls, it is suitable for tests that need
> >>> uniqueness
> >>> >> accross all nodes.
> >>> >>
> >>> >>> >> --
> >>> >>> >> Regards.
> >>> >>> >> Philippe M.
> >>> >>> >> @philmdot <https://twitter.com/philmdot>
> >>> >>>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Cordialement.
> >>> >> Philippe Mouawad.
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >
> >>> > --
> >>> > Cordialement.
> >>> > Philippe Mouawad.
> >>>
> >>
> >>
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >>
> >>
> >>
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by sebb <se...@gmail.com>.
On 13 November 2013 21:59, Philippe Mouawad <ph...@gmail.com> wrote:
> Hello sebb,
> Shall I commit this development or do we wait for 2.10.1 to be released ?
I am still concerned that this addition is specific to the Redis server.
I would much prefer to see a generic solution that can use various
different kinds of servers.
> When do you plan to commit your devs on keytool ?
Sorry, have not been able to spend any time on this recently.
> Thank you
> Regards
> Philippe
>
>
> On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <philippe.mouawad@gmail.com
>> wrote:
>
>> Hello Sebb,
>> My answers below.
>>
>> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
>>
>>> On 2 November 2013 08:28, Philippe Mouawad <ph...@gmail.com>
>>> wrote:
>>> > Hello,
>>> > Can I proceed with commit these up coming days ?
>>>
>>> I'm not sure that the discussion was completed.
>>>
>>> As far as I can tell, the proposal only suits some types of
>>> cloud-based test, and relies on 3rd party servers to hold the data.
>>>
>>
>> No in my opinion if fits many scenarios:
>> - Cloud based tests, this one seems to me an important one, as Cloud based
>> usage are increasing
>> - Distributed tests, even if it is possible to do it with CSV, having the
>> data in a remote server is much easier to manage than having to
>> split/distribute on servers. Even it is true it requires some skills to
>> manage correctly the Redis server
>> - Continuous integration tests where also having the data in a centralised
>> , remote servers is easier than managing CSVs. In this case Redis server
>> plays the same role as a JDBC repository
>> - Compared to a database it should perform better for high load tests
>> since it's an in-memory repo (although this can be done in SQL databases),
>> see http://redis.io/topics/benchmarks
>>
>>
>>> I'm not yet convinced that this is how we should be extending JMeter.
>>>
>>
>> I don't understand this argument, it would be another string for JMeter,
>> my implementation is only 2 classes + properties for I18N ?
>>
>> What do you propose in this case ?
>>
>>
>>
>>> > Thanks
>>> > Regards
>>> > Philippe
>>> >
>>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
>>> >
>>> >>
>>> >>
>>> >> On Monday, October 28, 2013, sebb wrote:
>>> >>
>>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
>>> philippe.mouawad@gmail.com>
>>> >>> wrote:
>>> >>> >> Hello,
>>> >>> >> These days Cloud based testing is becoming popular and having to
>>> >>> distribute
>>> >>> >> test data accross many servers through CSV can become painful if
>>> not
>>> >>> >> impossible.
>>> >>> >>
>>> >>> >> Even without Cloud, when using distributed testing you always have
>>> to
>>> >>> >> replicate your data on all servers, which is a painful manual step.
>>> >>> >>
>>> >>> >> Shouldn't we introduce a new DataSet more suitable for these use
>>> cases
>>> >>> ?
>>> >>> >>
>>> >>> >> We could do it in many different ways:
>>> >>> >> - Integrate an automatic CSV replicator, this would remain simple
>>> and
>>> >>> would
>>> >>> >> not introduce any tier, but with cloud based it would not
>>> horizontally
>>> >>> >> scale easily
>>> >>> >
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>> > Not sure I follow the scaling argument. The file would only have to
>>> be
>>> >>> > copied once before the test proper starts.
>>> >>> > From then on, the data is accessed locally.
>>> >>> >
>>> >>
>>> >>
>>> >> The scale word was not good, In my thinking the issue is more about
>>> >> deploying/copying/splitting the file among nodes or cloud members. A
>>> >> centralized access makes it far easier.
>>> >>
>>> >>
>>> >>> > However, with a database, each node will need at least one
>>> connection
>>> >>> > to the database, and every time more data is needed there will be
>>> >>> > network traffic.
>>> >>> > Or the database has to be running on the server node.
>>> >>> >
>>> >>
>>> >>
>>> >> Agree, I was not saying anything different. But as I said this can be
>>> >> useful for middle or low volume
>>> >>
>>> >>
>>> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs
>>> fine, and
>>> >>> >> jdbc protocol is not well suited for cloud based deployment (But it
>>> >>> could
>>> >>> >> also be an interesting feature for Continuous Integration)
>>> >>> >> - Use a NOSQL repository (Redis seems to me the best choice) , see
>>> >>> this
>>> >>> >> high level summary which I find interesting
>>> >>> >>
>>> >>>
>>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>>> >>> >>
>>> >>> >> I have implemented a new Redis (based on Java library for Redis)
>>> >>> DataSet
>>> >>> >> which I plan to commit if no objection.
>>> >>> >>
>>> >>> >> It will introduce 2 new dependencies with Apache License:
>>> >>> >> - Jedis (http://code.google.com/p/jedis/)
>>> >>> >> - commons-pool
>>> >>> >
>>> >>> > There is also a dependency on Redis, but I guess that would not be
>>> >>> bundled.
>>> >>>
>>> >>>
>>> >> no need to anything else than jedis + commons-pool
>>> >>
>>> >>
>>> >>> I've just noticed that Redis is provided as source which needs to be
>>> >>> built before use.
>>> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>>> >>> it's going to be much harder to install Redis.
>>> >>> In which case additional network traffic will be needed to access the
>>> >>> database.
>>> >>>
>>> >>> Also there is no official Windows release.
>>> >>>
>>> >>> >> Thoughts ?
>>> >>> >
>>> >>> > Is MongoDB not suitable?
>>> >>> > We already include a jar for it.
>>> >>> >
>>> >>
>>> >>
>>> >> Mongodb is more document oriented and has another type of use cases.
>>> One
>>> >> interesting feature of redis is lists where you can pop a line it will
>>> not
>>> >> be available to next calls, it is suitable for tests that need
>>> uniqueness
>>> >> accross all nodes.
>>> >>
>>> >>> >> --
>>> >>> >> Regards.
>>> >>> >> Philippe M.
>>> >>> >> @philmdot <https://twitter.com/philmdot>
>>> >>>
>>> >>
>>> >>
>>> >> --
>>> >> Cordialement.
>>> >> Philippe Mouawad.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >
>>> > --
>>> > Cordialement.
>>> > Philippe Mouawad.
>>>
>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello sebb,
Shall I commit this development or do we wait for 2.10.1 to be released ?
When do you plan to commit your devs on keytool ?
Thank you
Regards
Philippe
On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <philippe.mouawad@gmail.com
> wrote:
> Hello Sebb,
> My answers below.
>
> On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
>
>> On 2 November 2013 08:28, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>> > Hello,
>> > Can I proceed with commit these up coming days ?
>>
>> I'm not sure that the discussion was completed.
>>
>> As far as I can tell, the proposal only suits some types of
>> cloud-based test, and relies on 3rd party servers to hold the data.
>>
>
> No in my opinion if fits many scenarios:
> - Cloud based tests, this one seems to me an important one, as Cloud based
> usage are increasing
> - Distributed tests, even if it is possible to do it with CSV, having the
> data in a remote server is much easier to manage than having to
> split/distribute on servers. Even it is true it requires some skills to
> manage correctly the Redis server
> - Continuous integration tests where also having the data in a centralised
> , remote servers is easier than managing CSVs. In this case Redis server
> plays the same role as a JDBC repository
> - Compared to a database it should perform better for high load tests
> since it's an in-memory repo (although this can be done in SQL databases),
> see http://redis.io/topics/benchmarks
>
>
>> I'm not yet convinced that this is how we should be extending JMeter.
>>
>
> I don't understand this argument, it would be another string for JMeter,
> my implementation is only 2 classes + properties for I18N ?
>
> What do you propose in this case ?
>
>
>
>> > Thanks
>> > Regards
>> > Philippe
>> >
>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
>> >
>> >>
>> >>
>> >> On Monday, October 28, 2013, sebb wrote:
>> >>
>> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>> >>> > On 26 October 2013 20:23, Philippe Mouawad <
>> philippe.mouawad@gmail.com>
>> >>> wrote:
>> >>> >> Hello,
>> >>> >> These days Cloud based testing is becoming popular and having to
>> >>> distribute
>> >>> >> test data accross many servers through CSV can become painful if
>> not
>> >>> >> impossible.
>> >>> >>
>> >>> >> Even without Cloud, when using distributed testing you always have
>> to
>> >>> >> replicate your data on all servers, which is a painful manual step.
>> >>> >>
>> >>> >> Shouldn't we introduce a new DataSet more suitable for these use
>> cases
>> >>> ?
>> >>> >>
>> >>> >> We could do it in many different ways:
>> >>> >> - Integrate an automatic CSV replicator, this would remain simple
>> and
>> >>> would
>> >>> >> not introduce any tier, but with cloud based it would not
>> horizontally
>> >>> >> scale easily
>> >>> >
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>> > Not sure I follow the scaling argument. The file would only have to
>> be
>> >>> > copied once before the test proper starts.
>> >>> > From then on, the data is accessed locally.
>> >>> >
>> >>
>> >>
>> >> The scale word was not good, In my thinking the issue is more about
>> >> deploying/copying/splitting the file among nodes or cloud members. A
>> >> centralized access makes it far easier.
>> >>
>> >>
>> >>> > However, with a database, each node will need at least one
>> connection
>> >>> > to the database, and every time more data is needed there will be
>> >>> > network traffic.
>> >>> > Or the database has to be running on the server node.
>> >>> >
>> >>
>> >>
>> >> Agree, I was not saying anything different. But as I said this can be
>> >> useful for middle or low volume
>> >>
>> >>
>> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs
>> fine, and
>> >>> >> jdbc protocol is not well suited for cloud based deployment (But it
>> >>> could
>> >>> >> also be an interesting feature for Continuous Integration)
>> >>> >> - Use a NOSQL repository (Redis seems to me the best choice) , see
>> >>> this
>> >>> >> high level summary which I find interesting
>> >>> >>
>> >>>
>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>> >>> >>
>> >>> >> I have implemented a new Redis (based on Java library for Redis)
>> >>> DataSet
>> >>> >> which I plan to commit if no objection.
>> >>> >>
>> >>> >> It will introduce 2 new dependencies with Apache License:
>> >>> >> - Jedis (http://code.google.com/p/jedis/)
>> >>> >> - commons-pool
>> >>> >
>> >>> > There is also a dependency on Redis, but I guess that would not be
>> >>> bundled.
>> >>>
>> >>>
>> >> no need to anything else than jedis + commons-pool
>> >>
>> >>
>> >>> I've just noticed that Redis is provided as source which needs to be
>> >>> built before use.
>> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>> >>> it's going to be much harder to install Redis.
>> >>> In which case additional network traffic will be needed to access the
>> >>> database.
>> >>>
>> >>> Also there is no official Windows release.
>> >>>
>> >>> >> Thoughts ?
>> >>> >
>> >>> > Is MongoDB not suitable?
>> >>> > We already include a jar for it.
>> >>> >
>> >>
>> >>
>> >> Mongodb is more document oriented and has another type of use cases.
>> One
>> >> interesting feature of redis is lists where you can pop a line it will
>> not
>> >> be available to next calls, it is suitable for tests that need
>> uniqueness
>> >> accross all nodes.
>> >>
>> >>> >> --
>> >>> >> Regards.
>> >>> >> Philippe M.
>> >>> >> @philmdot <https://twitter.com/philmdot>
>> >>>
>> >>
>> >>
>> >> --
>> >> Cordialement.
>> >> Philippe Mouawad.
>> >>
>> >>
>> >>
>> >>
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello Sebb,
My answers below.
On Sat, Nov 2, 2013 at 10:35 AM, sebb <se...@gmail.com> wrote:
> On 2 November 2013 08:28, Philippe Mouawad <ph...@gmail.com>
> wrote:
> > Hello,
> > Can I proceed with commit these up coming days ?
>
> I'm not sure that the discussion was completed.
>
> As far as I can tell, the proposal only suits some types of
> cloud-based test, and relies on 3rd party servers to hold the data.
>
No in my opinion if fits many scenarios:
- Cloud based tests, this one seems to me an important one, as Cloud based
usage are increasing
- Distributed tests, even if it is possible to do it with CSV, having the
data in a remote server is much easier to manage than having to
split/distribute on servers. Even it is true it requires some skills to
manage correctly the Redis server
- Continuous integration tests where also having the data in a centralised
, remote servers is easier than managing CSVs. In this case Redis server
plays the same role as a JDBC repository
- Compared to a database it should perform better for high load tests since
it's an in-memory repo (although this can be done in SQL databases), see
http://redis.io/topics/benchmarks
> I'm not yet convinced that this is how we should be extending JMeter.
>
I don't understand this argument, it would be another string for JMeter, my
implementation is only 2 classes + properties for I18N ?
What do you propose in this case ?
> > Thanks
> > Regards
> > Philippe
> >
> > On Monday, October 28, 2013, Philippe Mouawad wrote:
> >
> >>
> >>
> >> On Monday, October 28, 2013, sebb wrote:
> >>
> >>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
> >>> > On 26 October 2013 20:23, Philippe Mouawad <
> philippe.mouawad@gmail.com>
> >>> wrote:
> >>> >> Hello,
> >>> >> These days Cloud based testing is becoming popular and having to
> >>> distribute
> >>> >> test data accross many servers through CSV can become painful if not
> >>> >> impossible.
> >>> >>
> >>> >> Even without Cloud, when using distributed testing you always have
> to
> >>> >> replicate your data on all servers, which is a painful manual step.
> >>> >>
> >>> >> Shouldn't we introduce a new DataSet more suitable for these use
> cases
> >>> ?
> >>> >>
> >>> >> We could do it in many different ways:
> >>> >> - Integrate an automatic CSV replicator, this would remain simple
> and
> >>> would
> >>> >> not introduce any tier, but with cloud based it would not
> horizontally
> >>> >> scale easily
> >>> >
> >>
> >>
> >>
> >>
> >>
> >>> > Not sure I follow the scaling argument. The file would only have to
> be
> >>> > copied once before the test proper starts.
> >>> > From then on, the data is accessed locally.
> >>> >
> >>
> >>
> >> The scale word was not good, In my thinking the issue is more about
> >> deploying/copying/splitting the file among nodes or cloud members. A
> >> centralized access makes it far easier.
> >>
> >>
> >>> > However, with a database, each node will need at least one connection
> >>> > to the database, and every time more data is needed there will be
> >>> > network traffic.
> >>> > Or the database has to be running on the server node.
> >>> >
> >>
> >>
> >> Agree, I was not saying anything different. But as I said this can be
> >> useful for middle or low volume
> >>
> >>
> >>> >> - Use a JDBC DataSet, but we would need to ensure it performs fine,
> and
> >>> >> jdbc protocol is not well suited for cloud based deployment (But it
> >>> could
> >>> >> also be an interesting feature for Continuous Integration)
> >>> >> - Use a NOSQL repository (Redis seems to me the best choice) , see
> >>> this
> >>> >> high level summary which I find interesting
> >>> >>
> >>>
> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
> >>> >>
> >>> >> I have implemented a new Redis (based on Java library for Redis)
> >>> DataSet
> >>> >> which I plan to commit if no objection.
> >>> >>
> >>> >> It will introduce 2 new dependencies with Apache License:
> >>> >> - Jedis (http://code.google.com/p/jedis/)
> >>> >> - commons-pool
> >>> >
> >>> > There is also a dependency on Redis, but I guess that would not be
> >>> bundled.
> >>>
> >>>
> >> no need to anything else than jedis + commons-pool
> >>
> >>
> >>> I've just noticed that Redis is provided as source which needs to be
> >>> built before use.
> >>> If it's difficult to copy CSV files to cloud nodes, it seems to me
> >>> it's going to be much harder to install Redis.
> >>> In which case additional network traffic will be needed to access the
> >>> database.
> >>>
> >>> Also there is no official Windows release.
> >>>
> >>> >> Thoughts ?
> >>> >
> >>> > Is MongoDB not suitable?
> >>> > We already include a jar for it.
> >>> >
> >>
> >>
> >> Mongodb is more document oriented and has another type of use cases.
> One
> >> interesting feature of redis is lists where you can pop a line it will
> not
> >> be available to next calls, it is suitable for tests that need
> uniqueness
> >> accross all nodes.
> >>
> >>> >> --
> >>> >> Regards.
> >>> >> Philippe M.
> >>> >> @philmdot <https://twitter.com/philmdot>
> >>>
> >>
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >>
> >>
> >>
> >>
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: New DataSet implementation
Posted by sebb <se...@gmail.com>.
On 2 November 2013 08:28, Philippe Mouawad <ph...@gmail.com> wrote:
> Hello,
> Can I proceed with commit these up coming days ?
I'm not sure that the discussion was completed.
As far as I can tell, the proposal only suits some types of
cloud-based test, and relies on 3rd party servers to hold the data.
I'm not yet convinced that this is how we should be extending JMeter.
> Thanks
> Regards
> Philippe
>
> On Monday, October 28, 2013, Philippe Mouawad wrote:
>
>>
>>
>> On Monday, October 28, 2013, sebb wrote:
>>
>>> On 28 October 2013 01:26, sebb <se...@gmail.com> wrote:
>>> > On 26 October 2013 20:23, Philippe Mouawad <ph...@gmail.com>
>>> wrote:
>>> >> Hello,
>>> >> These days Cloud based testing is becoming popular and having to
>>> distribute
>>> >> test data accross many servers through CSV can become painful if not
>>> >> impossible.
>>> >>
>>> >> Even without Cloud, when using distributed testing you always have to
>>> >> replicate your data on all servers, which is a painful manual step.
>>> >>
>>> >> Shouldn't we introduce a new DataSet more suitable for these use cases
>>> ?
>>> >>
>>> >> We could do it in many different ways:
>>> >> - Integrate an automatic CSV replicator, this would remain simple and
>>> would
>>> >> not introduce any tier, but with cloud based it would not horizontally
>>> >> scale easily
>>> >
>>
>>
>>
>>
>>
>>> > Not sure I follow the scaling argument. The file would only have to be
>>> > copied once before the test proper starts.
>>> > From then on, the data is accessed locally.
>>> >
>>
>>
>> The scale word was not good, In my thinking the issue is more about
>> deploying/copying/splitting the file among nodes or cloud members. A
>> centralized access makes it far easier.
>>
>>
>>> > However, with a database, each node will need at least one connection
>>> > to the database, and every time more data is needed there will be
>>> > network traffic.
>>> > Or the database has to be running on the server node.
>>> >
>>
>>
>> Agree, I was not saying anything different. But as I said this can be
>> useful for middle or low volume
>>
>>
>>> >> - Use a JDBC DataSet, but we would need to ensure it performs fine, and
>>> >> jdbc protocol is not well suited for cloud based deployment (But it
>>> could
>>> >> also be an interesting feature for Continuous Integration)
>>> >> - Use a NOSQL repository (Redis seems to me the best choice) , see
>>> this
>>> >> high level summary which I find interesting
>>> >>
>>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>>> >>
>>> >> I have implemented a new Redis (based on Java library for Redis)
>>> DataSet
>>> >> which I plan to commit if no objection.
>>> >>
>>> >> It will introduce 2 new dependencies with Apache License:
>>> >> - Jedis (http://code.google.com/p/jedis/)
>>> >> - commons-pool
>>> >
>>> > There is also a dependency on Redis, but I guess that would not be
>>> bundled.
>>>
>>>
>> no need to anything else than jedis + commons-pool
>>
>>
>>> I've just noticed that Redis is provided as source which needs to be
>>> built before use.
>>> If it's difficult to copy CSV files to cloud nodes, it seems to me
>>> it's going to be much harder to install Redis.
>>> In which case additional network traffic will be needed to access the
>>> database.
>>>
>>> Also there is no official Windows release.
>>>
>>> >> Thoughts ?
>>> >
>>> > Is MongoDB not suitable?
>>> > We already include a jar for it.
>>> >
>>
>>
>> Mongodb is more document oriented and has another type of use cases. One
>> interesting feature of redis is lists where you can pop a line it will not
>> be available to next calls, it is suitable for tests that need uniqueness
>> accross all nodes.
>>
>>> >> --
>>> >> Regards.
>>> >> Philippe M.
>>> >> @philmdot <https://twitter.com/philmdot>
>>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>>
>
> --
> Cordialement.
> Philippe Mouawad.