You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Pablo Estrada <pa...@google.com> on 2018/08/14 22:27:19 UTC
Policy for Python ValidatesRunner vs IT tests?
Hello,
In Python, we tag some test methods with @attr('ValidatesRunner') and
@attr('IT'), which marks them to be run as pipeline tests.
If I understand correctly:
- ValidatesRunner tests are more like a component tests[1] as explained in
Beam docs
- IT tests are more like a E2E test[2] as explained in the docs. Is there
an equivalent in Java?
- Finally, there's ValidatesContainer tests. What are these for? What's the
guidance for tagging our tests this way?
Thanks!
-P.
[1] https://beam.apache.org/contribute/testing/#validatesrunner
[2] https://beam.apache.org/contribute/testing/#e2e
--
Got feedback? go/pabloem-feedback
Re: Policy for Python ValidatesRunner vs IT tests?
Posted by Robin Qiu <ro...@google.com>.
Hi Pablo,
Beam Java has unit tests, ValidatesRunner tests, and Integration (E2E) test.
If I understand correctly, unit tests test the functionality of a component
and usually don't require a runner (is this true?); ValidatesRunner tests
(marked with @ValidatesRunner annotation) require runners; and integration
tests (named *IT.java) test an entire e2e pipeline.
Best,
Robin
On Tue, Aug 14, 2018 at 3:39 PM Pablo Estrada <pa...@google.com> wrote:
> Ah this is very helpful. I think this is missing in the testing guide.
> I'll make a short PR to mention it.
> Best
> -P.
>
> On Tue, Aug 14, 2018 at 3:37 PM Anton Kedin <ke...@google.com> wrote:
>
>> IT tests exist in java, similar to unit tests and not marked in a special
>> way, except they're called *IT.java instead of *Test.java. They're run from
>> corresponding tasks:
>> -
>> https://github.com/apache/beam/blob/d6c5bf977fc688f289f1bb06e30f25b05bf987b2/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubReadIT.java#L33
>>
>> -
>> https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/build.gradle#L85
>>
>> On Tue, Aug 14, 2018 at 3:27 PM Pablo Estrada <pa...@google.com> wrote:
>>
>>> Hello,
>>> In Python, we tag some test methods with @attr('ValidatesRunner') and
>>> @attr('IT'), which marks them to be run as pipeline tests.
>>>
>>> If I understand correctly:
>>> - ValidatesRunner tests are more like a component tests[1] as explained
>>> in Beam docs
>>> - IT tests are more like a E2E test[2] as explained in the docs. Is
>>> there an equivalent in Java?
>>> - Finally, there's ValidatesContainer tests. What are these for? What's
>>> the guidance for tagging our tests this way?
>>>
>>> Thanks!
>>> -P.
>>>
>>> [1] https://beam.apache.org/contribute/testing/#validatesrunner
>>> [2] https://beam.apache.org/contribute/testing/#e2e
>>> --
>>> Got feedback? go/pabloem-feedback
>>> <https://goto.google.com/pabloem-feedback>
>>>
>> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>
Re: Policy for Python ValidatesRunner vs IT tests?
Posted by Pablo Estrada <pa...@google.com>.
Ah this is very helpful. I think this is missing in the testing guide. I'll
make a short PR to mention it.
Best
-P.
On Tue, Aug 14, 2018 at 3:37 PM Anton Kedin <ke...@google.com> wrote:
> IT tests exist in java, similar to unit tests and not marked in a special
> way, except they're called *IT.java instead of *Test.java. They're run from
> corresponding tasks:
> -
> https://github.com/apache/beam/blob/d6c5bf977fc688f289f1bb06e30f25b05bf987b2/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubReadIT.java#L33
>
> -
> https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/build.gradle#L85
>
> On Tue, Aug 14, 2018 at 3:27 PM Pablo Estrada <pa...@google.com> wrote:
>
>> Hello,
>> In Python, we tag some test methods with @attr('ValidatesRunner') and
>> @attr('IT'), which marks them to be run as pipeline tests.
>>
>> If I understand correctly:
>> - ValidatesRunner tests are more like a component tests[1] as explained
>> in Beam docs
>> - IT tests are more like a E2E test[2] as explained in the docs. Is there
>> an equivalent in Java?
>> - Finally, there's ValidatesContainer tests. What are these for? What's
>> the guidance for tagging our tests this way?
>>
>> Thanks!
>> -P.
>>
>> [1] https://beam.apache.org/contribute/testing/#validatesrunner
>> [2] https://beam.apache.org/contribute/testing/#e2e
>> --
>> Got feedback? go/pabloem-feedback
>> <https://goto.google.com/pabloem-feedback>
>>
> --
Got feedback? go/pabloem-feedback
Re: Policy for Python ValidatesRunner vs IT tests?
Posted by Anton Kedin <ke...@google.com>.
IT tests exist in java, similar to unit tests and not marked in a special
way, except they're called *IT.java instead of *Test.java. They're run from
corresponding tasks:
-
https://github.com/apache/beam/blob/d6c5bf977fc688f289f1bb06e30f25b05bf987b2/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubReadIT.java#L33
-
https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/build.gradle#L85
On Tue, Aug 14, 2018 at 3:27 PM Pablo Estrada <pa...@google.com> wrote:
> Hello,
> In Python, we tag some test methods with @attr('ValidatesRunner') and
> @attr('IT'), which marks them to be run as pipeline tests.
>
> If I understand correctly:
> - ValidatesRunner tests are more like a component tests[1] as explained in
> Beam docs
> - IT tests are more like a E2E test[2] as explained in the docs. Is there
> an equivalent in Java?
> - Finally, there's ValidatesContainer tests. What are these for? What's
> the guidance for tagging our tests this way?
>
> Thanks!
> -P.
>
> [1] https://beam.apache.org/contribute/testing/#validatesrunner
> [2] https://beam.apache.org/contribute/testing/#e2e
> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>
Re: Policy for Python ValidatesRunner vs IT tests?
Posted by Alan Myrvold <am...@google.com>.
The ValidatesContainer test builds the python portable worker container
using docker, pushes it, then runs a dataflow pipeline using the container
with the flag --worker_harness_container_image. There is only one test that
does this, and it is also an 'IT' test.
On Tue, Aug 14, 2018 at 3:27 PM Pablo Estrada <pa...@google.com> wrote:
> Hello,
> In Python, we tag some test methods with @attr('ValidatesRunner') and
> @attr('IT'), which marks them to be run as pipeline tests.
>
> If I understand correctly:
> - ValidatesRunner tests are more like a component tests[1] as explained in
> Beam docs
> - IT tests are more like a E2E test[2] as explained in the docs. Is there
> an equivalent in Java?
> - Finally, there's ValidatesContainer tests. What are these for? What's
> the guidance for tagging our tests this way?
>
> Thanks!
> -P.
>
> [1] https://beam.apache.org/contribute/testing/#validatesrunner
> [2] https://beam.apache.org/contribute/testing/#e2e
> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>