is integration testing of custom serializer code without running a multi-node cluster possible?

[Chris Bedford <ch...@buildlackey.com>]

Hi there -

Is integration testing of custom serializer code without running a
multi-node cluster possible?
I am asking because I need to integration test  various customer
serializers we are writing for our current  project.
As I understand it, only if a spout or bolt is sending a tuple to a tuple
in another JVM does kryo serialization occur.
I'd like to do some unit tests and force that serialization to happen even
if I am running everything in
memory  in one JVM   (via LocalCluster).


Are there any tricks to do this?   Or any other tips that I should keep in
mind ?

thanks very much in advance for your help
 - chris





-- 
Chris Bedford

Founder & Lead Lackey
Build Lackey Labs:  http://buildlackey.com
Go Grails!: http://blog.buildlackey.com

Re: is integration testing of custom serializer code without running a multi-node cluster possible?

[Guillaume Perrot <gp...@ubikod.com>]

Hi,
Local mode can also work with multiple workers, just increase the number of
workers in the conf object you submit to LocalCluster.
Raising parallelism hint of components will also raise the chances Storm
need to serialize tuples.
I tested myself and my serializer is called in local mode (not for all
tuples of course).

One trick I did not try but should work: 2 workers, 1 spout (no parallel) 1
bolt (no parallel) and every tuple should be serialized in that case.

2013/12/18 Chris Bedford <ch...@buildlackey.com>

>
>
> Hi there -
>
> Is integration testing of custom serializer code without running a
> multi-node cluster possible?
> I am asking because I need to integration test  various customer
> serializers we are writing for our current  project.
> As I understand it, only if a spout or bolt is sending a tuple to a tuple
> in another JVM does kryo serialization occur.
> I'd like to do some unit tests and force that serialization to happen even
> if I am running everything in
> memory  in one JVM   (via LocalCluster).
>
>
> Are there any tricks to do this?   Or any other tips that I should keep in
> mind ?
>
> thanks very much in advance for your help
>   - chris
>
>
>
>
>
> --
> Chris Bedford
>
> Founder & Lead Lackey
> Build Lackey Labs:  http://buildlackey.com
> Go Grails!: http://blog.buildlackey.com
>