Go SDK integration tests

[Henning Rohde <he...@google.com>]

Hi everyone,

 I'm currently tinkering with adding integration tests for Go (BEAM-3827)
and wrote down a small proposal to that end:

https://docs.google.com/document/d/1jy6EE7D4RjgfNV0FhD3rMsT1YKhnUfcHRZMAlC6ygXw/edit?usp=sharing

Similarly to other SDKs, the proposal is to add self-validating integration
tests that don't produce output. But unlike Java, we can't easily reuse the
Go example code directly and use a driver program with all tests linked in
to run against an arbitrary runner.

Comments welcome!

Thanks,
 Henning

Re: Go SDK integration tests

[Henning Rohde <he...@google.com>]

Thanks Kenn. For now, the idea is just to have something (wordcount!)
running -- but the Go integration test suite should ideally cover the whole
Go implementation. The difference from ValidatesRunner tests is that it
targets SDK harness coverage, not runner coverage, although there is
substantial overlap. In practice, it'll likely serve both purposes.

On Mon, May 14, 2018 at 1:34 PM Kenneth Knowles <kl...@google.com> wrote:

> Nice! This is super important. Commented a bit. Any thoughts on the scope
> of tests you want to add? You mention not as extensive as the Java
> ValidatesRunner, which makes sense. So are you mostly targeting the basic
> example ITs as in the example in the doc?
>
> On the other hand, I think a Golang-based ValidatesRunner suite might do
> an even better job of keeping portable runners from accidentally using
> shortcuts.
>
> Kenn
>
> On Tue, May 8, 2018 at 6:14 PM Henning Rohde <he...@google.com> wrote:
>
>> Hi everyone,
>>
>>  I'm currently tinkering with adding integration tests for Go (BEAM-3827)
>> and wrote down a small proposal to that end:
>>
>>
>> https://docs.google.com/document/d/1jy6EE7D4RjgfNV0FhD3rMsT1YKhnUfcHRZMAlC6ygXw/edit?usp=sharing
>>
>> Similarly to other SDKs, the proposal is to add self-validating
>> integration tests that don't produce output. But unlike Java, we can't
>> easily reuse the Go example code directly and use a driver program with all
>> tests linked in to run against an arbitrary runner.
>>
>> Comments welcome!
>>
>> Thanks,
>>  Henning
>>
>>

Re: Go SDK integration tests

[Kenneth Knowles <kl...@google.com>]

Nice! This is super important. Commented a bit. Any thoughts on the scope
of tests you want to add? You mention not as extensive as the Java
ValidatesRunner, which makes sense. So are you mostly targeting the basic
example ITs as in the example in the doc?

On the other hand, I think a Golang-based ValidatesRunner suite might do an
even better job of keeping portable runners from accidentally using
shortcuts.

Kenn

On Tue, May 8, 2018 at 6:14 PM Henning Rohde <he...@google.com> wrote:

> Hi everyone,
>
>  I'm currently tinkering with adding integration tests for Go (BEAM-3827)
> and wrote down a small proposal to that end:
>
>
> https://docs.google.com/document/d/1jy6EE7D4RjgfNV0FhD3rMsT1YKhnUfcHRZMAlC6ygXw/edit?usp=sharing
>
> Similarly to other SDKs, the proposal is to add self-validating
> integration tests that don't produce output. But unlike Java, we can't
> easily reuse the Go example code directly and use a driver program with all
> tests linked in to run against an arbitrary runner.
>
> Comments welcome!
>
> Thanks,
>  Henning
>
>