You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geode.apache.org by Real Wes Williams <Th...@outlook.com> on 2016/04/10 06:28:46 UTC
Re: Unit Testing and TDD with Geode
I have picked up this thread and am pursuing it with interest.
I am not using SDG but I _am_ using both a peer and a peer cache server to drive the test, per John’s idea to use a peer. Executing as a peer _should_ work. The debugger stops in the function. There is no need to mock because the arguments are legitimately being passed into the function and the debugger breakpoint stops inside the function.
All attempts have failed against executing .onRegion(filter) with Partitioned regions because once in the function, the regionFunctionContext.getDataSet is empty. This is true whether local-max-memory=“0” is present or not.
The empty region makes sense if I do _not_ have local-max-memory=“0” since I would need to do a rebalance before the peer _server_ region has data. However, I don’t see why that would be true in the case of a peer with local-max-memory=“0”. It’s a peer. It has no data. If I execute with a set of filter keys, the threads of execution should go to the nodes that have the keys and not the peer; since the peer does not have the key by definition. Since this is so, why is the thread of execution inside the function showing that the region is empty?
That being said, the goal is to unit test the function and should I not be able to do that with a peer? If not, is there a non SDG example of what needs to be mocked?
Thanks,
Wes Williams
> On Mar 3, 2016, at 1:22 PM, Dan Smith <ds...@pivotal.io> wrote:
>
> Hi Matt,
>
> For unit testing Functions and cache listeners, you basically just need to want to mock the arguments. Mockito or jMock are good choices for creating a mock FunctionContext or EntryEvent. You hopefully shouldn't need to use PowerMock - if you run into a case where you think you need it, please file a bug against geode API!
>
> One option to consider is whether you want to use spring data gemfire's annotation support for functions. That can avoid the need to mock a function context, because spring will automatically map the arguments to the parameters of your method.
>
> http://docs.spring.io/spring-data-gemfire/docs/current/reference/html/#function-annotations <http://docs.spring.io/spring-data-gemfire/docs/current/reference/html/#function-annotations>
>
> On Thu, Mar 3, 2016 at 8:56 AM, Matt Ross <mross@pivotal.io <ma...@pivotal.io>> wrote:
> Ethan,
>
> The concept of what I'm trying to unit test is not what I'm asking about. I know it's not the developers job to unit test whether a cache listener properly fires. I'm saying let's say we have a cache listener that takes events and does a transformation on them. Or I have a function that returns to me the results of a specific Oql query. I'm asking how from a technical perspective am I able to write unit tests so that I could run them without bringing up a cluster.
>
> My ideal workflow involves being able to run these unit tests as part of my Jenkins CI pipeline.
>
>
> On Thursday, March 3, 2016, Eitan Suez <esuez@pivotal.io <ma...@pivotal.io>> wrote:
> hi matt,
>
> what helps me is to ask myself "what is it i want to test?"
>
> i should be confident, for example, that gemfire will call my cachelistener on a cacheevent. so i shouldn't need to test that (at least not at the unit level). but whether the code in the body of my afterCreate() or afterUpdate() is doing what i expect it to is my responsibility: that's what i need to unit test. so technically you shouldn't have to bring in the gemfire machinery to unit-test a cachelistener.
>
> now if the body of that method interacts with regions (for example), you'll want to mock that interaction.
>
> integration or functional testing is more involved: you have to stand up an instance of gemfire with the configuration that will install your cache listeners or functions, and have to setup some trigger that will exercise the function you're testing, and then be able to verify the outcome.
>
> / eitan
>
>
> On Thu, Mar 3, 2016 at 9:06 AM, Matt Ross <mross@pivotal.io <>> wrote:
> Hi all
>
> A colleague and I have been talking about the best strategies to do unit testing and TDD in Gemfire/Geode. Obviously writing unit tests for a Gemfire/Geode Client is fairly straightforward, but how would I unit test a function that I'm running on the cluster. Or how would I write tests for a cache listener? I've been pointed in the direction of power mock but could still use some more input from the community on what some of the better strategies are.
>
> Are there any examples on Github of how to write these serverside unit tests? Like I said at this point we're only able to do integration testing and unit testing of the client so I've been fairly limited in being able to do TDD. Thanks!
>
> --
> Matthew Ross | Data Engineer | Pivotal
> 625 Avenue of the Americas NY, NY 10011
> 516-941-7535 <tel:516-941-7535> | mross@pivotal.io <>
>
>
>
>
> --
> Matthew Ross | Data Engineer | Pivotal
> 625 Avenue of the Americas NY, NY 10011
> 516-941-7535 <tel:516-941-7535> | mross@pivotal.io <ma...@pivotal.io>
>
>
>
Re: Unit Testing and TDD with Geode
Posted by Real Wes Williams <Th...@outlook.com>.
Hey! The instant I woke up this morning the answer popped into my head. Tested it and it all works now. I wanted to post before John goes through all of the helpful essay work in reply.
For those reading this thread in the future:
The peer is not part of the same distributed system as the running distributed system (locator=). So when the unit test runs as a peer/ server, it has no data. I connect to a remote DS, copy over data to my newly created peer/server and everything works great. Or in your case, just populate the peer/server with test data.
This was a helpful and practical thread.
Wes
> On Apr 10, 2016, at 12:28 AM, Real Wes Williams <Th...@outlook.com> wrote:
>
> I have picked up this thread and am pursuing it with interest.
>
> I am not using SDG but I _am_ using both a peer and a peer cache server to drive the test, per John’s idea to use a peer. Executing as a peer _should_ work. The debugger stops in the function. There is no need to mock because the arguments are legitimately being passed into the function and the debugger breakpoint stops inside the function.
>
> All attempts have failed against executing .onRegion(filter) with Partitioned regions because once in the function, the regionFunctionContext.getDataSet is empty. This is true whether local-max-memory=“0” is present or not.
>
> The empty region makes sense if I do _not_ have local-max-memory=“0” since I would need to do a rebalance before the peer _server_ region has data. However, I don’t see why that would be true in the case of a peer with local-max-memory=“0”. It’s a peer. It has no data. If I execute with a set of filter keys, the threads of execution should go to the nodes that have the keys and not the peer; since the peer does not have the key by definition. Since this is so, why is the thread of execution inside the function showing that the region is empty?
>
> That being said, the goal is to unit test the function and should I not be able to do that with a peer? If not, is there a non SDG example of what needs to be mocked?
>
> Thanks,
> Wes Williams
>
>
>
>> On Mar 3, 2016, at 1:22 PM, Dan Smith <dsmith@pivotal.io <ma...@pivotal.io>> wrote:
>>
>> Hi Matt,
>>
>> For unit testing Functions and cache listeners, you basically just need to want to mock the arguments. Mockito or jMock are good choices for creating a mock FunctionContext or EntryEvent. You hopefully shouldn't need to use PowerMock - if you run into a case where you think you need it, please file a bug against geode API!
>>
>> One option to consider is whether you want to use spring data gemfire's annotation support for functions. That can avoid the need to mock a function context, because spring will automatically map the arguments to the parameters of your method.
>>
>> http://docs.spring.io/spring-data-gemfire/docs/current/reference/html/#function-annotations <http://docs.spring.io/spring-data-gemfire/docs/current/reference/html/#function-annotations>
>>
>> On Thu, Mar 3, 2016 at 8:56 AM, Matt Ross <mross@pivotal.io <ma...@pivotal.io>> wrote:
>> Ethan,
>>
>> The concept of what I'm trying to unit test is not what I'm asking about. I know it's not the developers job to unit test whether a cache listener properly fires. I'm saying let's say we have a cache listener that takes events and does a transformation on them. Or I have a function that returns to me the results of a specific Oql query. I'm asking how from a technical perspective am I able to write unit tests so that I could run them without bringing up a cluster.
>>
>> My ideal workflow involves being able to run these unit tests as part of my Jenkins CI pipeline.
>>
>>
>> On Thursday, March 3, 2016, Eitan Suez <esuez@pivotal.io <ma...@pivotal.io>> wrote:
>> hi matt,
>>
>> what helps me is to ask myself "what is it i want to test?"
>>
>> i should be confident, for example, that gemfire will call my cachelistener on a cacheevent. so i shouldn't need to test that (at least not at the unit level). but whether the code in the body of my afterCreate() or afterUpdate() is doing what i expect it to is my responsibility: that's what i need to unit test. so technically you shouldn't have to bring in the gemfire machinery to unit-test a cachelistener.
>>
>> now if the body of that method interacts with regions (for example), you'll want to mock that interaction.
>>
>> integration or functional testing is more involved: you have to stand up an instance of gemfire with the configuration that will install your cache listeners or functions, and have to setup some trigger that will exercise the function you're testing, and then be able to verify the outcome.
>>
>> / eitan
>>
>>
>> On Thu, Mar 3, 2016 at 9:06 AM, Matt Ross <mross@pivotal.io <>> wrote:
>> Hi all
>>
>> A colleague and I have been talking about the best strategies to do unit testing and TDD in Gemfire/Geode. Obviously writing unit tests for a Gemfire/Geode Client is fairly straightforward, but how would I unit test a function that I'm running on the cluster. Or how would I write tests for a cache listener? I've been pointed in the direction of power mock but could still use some more input from the community on what some of the better strategies are.
>>
>> Are there any examples on Github of how to write these serverside unit tests? Like I said at this point we're only able to do integration testing and unit testing of the client so I've been fairly limited in being able to do TDD. Thanks!
>>
>> --
>> Matthew Ross | Data Engineer | Pivotal
>> 625 Avenue of the Americas NY, NY 10011
>> 516-941-7535 <tel:516-941-7535> | mross@pivotal.io <>
>>
>>
>>
>>
>> --
>> Matthew Ross | Data Engineer | Pivotal
>> 625 Avenue of the Americas NY, NY 10011
>> 516-941-7535 <tel:516-941-7535> | mross@pivotal.io <ma...@pivotal.io>
>>
>>
>>
>