Samza Unit Test Instrucations

[Yuanchi Ning <yu...@uber.com>]

Hello Samza Team,

This is Yuanchi Ning from Uber Data Engineering, Realtime Metrics,
Streaming Platform team.

We are planning to use Samza to process the realtime data we have, and
thanks for developing such an awesome open source project.

While I am building our streaming service using Samza, I am wondering is
there anyway to do unit tests for each Samza application Task instead of
integration test? Say, set up the mocker environment for feeding data into
MessageEnvelope or something similar?

Thanks for your assistance.

Best,
Yuanchi

Re: Samza Unit Test Instrucations

[Yan Fang <ya...@gmail.com>]

Hi Yuanchi,

There is no out-of-box unit tests provided by Samza. But there are some
ways:

1) If you only want to test the logic in the Task class, normal unit tests
will work. You can create a unit test that tests init(), process(), etc.

2) Create mock systems by implementing SystemAdmin, SystemConsumer, etc.
You can refer to this one TestSystemStreamPartitionIterator
<https://github.com/apache/samza/blob/master/samza-api/src/test/java/org/apache/samza/system/TestSystemStreamPartitionIterator.java>

3) bring up zookeeper, kafka in the unit test. Here is the reference
TestStatefulTask
<https://github.com/apache/samza/blob/master/samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala>

4) run the job locally
<http://samza.apache.org/learn/documentation/0.9/jobs/job-runner.html> by
setting the job.factory.class
<http://samza.apache.org/learn/documentation/0.9/jobs/configuration-table.html>
.

Usually, I think 1) and 4) are sufficient. Hope this helps.

Thanks,

Fang, Yan
yanfang724@gmail.com

On Wed, Apr 15, 2015 at 1:47 PM, Yuanchi Ning <yu...@uber.com> wrote:

> Hello Samza Team,
>
> This is Yuanchi Ning from Uber Data Engineering, Realtime Metrics,
> Streaming Platform team.
>
> We are planning to use Samza to process the realtime data we have, and
> thanks for developing such an awesome open source project.
>
> While I am building our streaming service using Samza, I am wondering is
> there anyway to do unit tests for each Samza application Task instead of
> integration test? Say, set up the mocker environment for feeding data into
> MessageEnvelope or something similar?
>
> Thanks for your assistance.
>
> Best,
> Yuanchi
>