Re: [BEAM-9015] Adding pyXX-cloud instead of pyXX-gcp and pyXX-aws

[Pablo Estrada <pa...@google.com>]

now back from the holidays, I intend to do this - one of these days.....

On Thu, Dec 26, 2019 at 12:51 PM Udi Meiri <eh...@google.com> wrote:

> +1
>
> On Mon, Dec 23, 2019, 17:28 Robert Bradshaw <ro...@google.com> wrote:
>
>> Makes sense to me.
>>
>> On Mon, Dec 23, 2019 at 3:33 PM Pablo Estrada <pa...@google.com> wrote:
>> >
>> > Hi all,
>> > a couple of contributors [1][2] have been kind enough to add support
>> for s3 filesystem[3] for the Python SDK. Part of this involved adding a tox
>> task called py37-aws, to install the relevant dependencies and run unit
>> tests for it (in a mocked-out environment).
>> >
>> > To avoid running a full extra test suite, I thought we could add the
>> new aws-related dependencies to the current pyXX-gcp suites, and perhaps
>> rename to pyXX-cloud, to include all unit tests that require cloud-specific
>> dependencies. What do others think?
>> >
>> > This is tracked here: https://jira.apache.org/jira/browse/BEAM-9015
>> >
>> > [1] https://github.com/tamera-lanham
>> > [2] https://github.com/MattMorgis
>> > [3] https://github.com/apache/beam/pull/9955
>>
>

Re: [BEAM-9015] Adding pyXX-cloud instead of pyXX-gcp and pyXX-aws

[Udi Meiri <eh...@google.com>]

We would like to run unit tests using these dependencies (gcp+aws) in
presubmits.
Having separate tox environments for each would require running tox for
each, increasing presubmit time and duplicating work (since most tests
don't depend on aws or gcp).


On Wed, Jan 15, 2020 at 1:54 PM Kyle Weaver <kc...@google.com> wrote:

> Just now seeing this -- are we sure we want to mix the signal from what
> are logically two totally separate test suites?
> Or from the opposite perspective, what's the motivation for wanting one
> test suite instead of two?
>
> On Tue, Jan 14, 2020 at 3:25 PM Pablo Estrada <pa...@google.com> wrote:
>
>> now back from the holidays, I intend to do this - one of these days.....
>>
>> On Thu, Dec 26, 2019 at 12:51 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> +1
>>>
>>> On Mon, Dec 23, 2019, 17:28 Robert Bradshaw <ro...@google.com> wrote:
>>>
>>>> Makes sense to me.
>>>>
>>>> On Mon, Dec 23, 2019 at 3:33 PM Pablo Estrada <pa...@google.com>
>>>> wrote:
>>>> >
>>>> > Hi all,
>>>> > a couple of contributors [1][2] have been kind enough to add support
>>>> for s3 filesystem[3] for the Python SDK. Part of this involved adding a tox
>>>> task called py37-aws, to install the relevant dependencies and run unit
>>>> tests for it (in a mocked-out environment).
>>>> >
>>>> > To avoid running a full extra test suite, I thought we could add the
>>>> new aws-related dependencies to the current pyXX-gcp suites, and perhaps
>>>> rename to pyXX-cloud, to include all unit tests that require cloud-specific
>>>> dependencies. What do others think?
>>>> >
>>>> > This is tracked here: https://jira.apache.org/jira/browse/BEAM-9015
>>>> >
>>>> > [1] https://github.com/tamera-lanham
>>>> > [2] https://github.com/MattMorgis
>>>> > [3] https://github.com/apache/beam/pull/9955
>>>>
>>>

Re: [BEAM-9015] Adding pyXX-cloud instead of pyXX-gcp and pyXX-aws

[Kyle Weaver <kc...@google.com>]

Just now seeing this -- are we sure we want to mix the signal from what are
logically two totally separate test suites?
Or from the opposite perspective, what's the motivation for wanting one
test suite instead of two?

On Tue, Jan 14, 2020 at 3:25 PM Pablo Estrada <pa...@google.com> wrote:

> now back from the holidays, I intend to do this - one of these days.....
>
> On Thu, Dec 26, 2019 at 12:51 PM Udi Meiri <eh...@google.com> wrote:
>
>> +1
>>
>> On Mon, Dec 23, 2019, 17:28 Robert Bradshaw <ro...@google.com> wrote:
>>
>>> Makes sense to me.
>>>
>>> On Mon, Dec 23, 2019 at 3:33 PM Pablo Estrada <pa...@google.com>
>>> wrote:
>>> >
>>> > Hi all,
>>> > a couple of contributors [1][2] have been kind enough to add support
>>> for s3 filesystem[3] for the Python SDK. Part of this involved adding a tox
>>> task called py37-aws, to install the relevant dependencies and run unit
>>> tests for it (in a mocked-out environment).
>>> >
>>> > To avoid running a full extra test suite, I thought we could add the
>>> new aws-related dependencies to the current pyXX-gcp suites, and perhaps
>>> rename to pyXX-cloud, to include all unit tests that require cloud-specific
>>> dependencies. What do others think?
>>> >
>>> > This is tracked here: https://jira.apache.org/jira/browse/BEAM-9015
>>> >
>>> > [1] https://github.com/tamera-lanham
>>> > [2] https://github.com/MattMorgis
>>> > [3] https://github.com/apache/beam/pull/9955
>>>
>>