You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Łukasz Gajowy <lg...@apache.org> on 2019/08/28 14:10:51 UTC
Stop publishing unneeded Java artifacts
Hi all,
I wanted to notify that in PR 9417
<https://github.com/apache/beam/pull/9417> I'm planning to turn off
publishing of the following modules' artifacts to the maven repository:
- :runners:google-cloud-dataflow-java:worker:windmill
- :sdks:java:build-tools
- :sdks:java:javadoc
- :sdks:java:testing:expansion-service
- :sdks:java:io:bigquery-io-perf-tests
- :sdks:java:io:file-based-io-tests
- :sdks:java:io:elasticsearch-tests:elasticsearch-tests-2
- :sdks:java:io:elasticsearch-tests:elasticsearch-tests-5
- :sdks:java:io:elasticsearch-tests:elasticsearch-tests-6
- :sdks:java:io:elasticsearch-tests:elasticsearch-tests-common
- :sdks:java:testing:load-tests
- :sdks:java:testing:nexmark
- :sdks:java:testing:test-utils
AFAIK, the purpose of these modules is to keep related
tests/test-utils/utils together. We are not expecting users to make use of
such artifacts. Please let me know if you have any objections. If there are
none and the PR gets merged, the artifacts will no longer be published.
Thanks!
Łukasz
Fwd: Stop publishing unneeded Java artifacts
Posted by Byung-Gon Chun <bg...@gmail.com>.
FYI.
전달된 메시지 시작:
> 보낸 사람: Kenneth Knowles <ke...@apache.org>
> 날짜: 2019년 9월 4일 오전 5시 19분 3초 GMT+9
> 받는 사람: dev <de...@beam.apache.org>
> 제목: Re: Stop publishing unneeded Java artifacts
> 답장받는 사람: dev@beam.apache.org
>
> I know that the Nemo project had tested the Nexmark pipelines. They may only need the main code, not the tests themselves. But the tests check that the runner's output matches a straight-line Java model of the query. So that is actually a pretty good runner test.
>
> I think any test suite that uses TestPipeline is potentially a useful integration test jar for runner developed outside our repository.
>
> Kenn
>
>> On Tue, Sep 3, 2019 at 10:43 AM Robert Bradshaw <ro...@google.com> wrote:
>> :sdks:java:testing:expansion-service could be useful to publish for
>> testing as well.
>>
>> On Fri, Aug 30, 2019 at 3:13 PM Lukasz Cwik <lc...@google.com> wrote:
>> >
>> > Google internally relies on being able to get the POM files generated for:
>> > :sdks:java:testing:nexmark
>> > :sdks:java:testing:test-utils
>> >
>> > Generating the POM files currently relies on publishing being enabled for those projects so could we keep publishing those two modules. Disabling the others sounds fine to me.
>> >
>> > On Thu, Aug 29, 2019 at 9:41 PM Lukasz Cwik <lc...@google.com> wrote:
>> >>
>> >> I wanted to double check that we don't rely on this publishing inside Google for some reason. Will update this thread tomorrow.
>> >>
>> >> On Wed, Aug 28, 2019 at 7:11 AM Łukasz Gajowy <lg...@apache.org> wrote:
>> >>>
>> >>> Hi all,
>> >>>
>> >>> I wanted to notify that in PR 9417 I'm planning to turn off publishing of the following modules' artifacts to the maven repository:
>> >>>
>> >>> :runners:google-cloud-dataflow-java:worker:windmill
>> >>> :sdks:java:build-tools
>> >>> :sdks:java:javadoc
>> >>> :sdks:java:testing:expansion-service
>> >>> :sdks:java:io:bigquery-io-perf-tests
>> >>> :sdks:java:io:file-based-io-tests
>> >>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-2
>> >>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-5
>> >>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-6
>> >>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-common
>> >>> :sdks:java:testing:load-tests
>> >>> :sdks:java:testing:nexmark
>> >>> :sdks:java:testing:test-utils
>> >>>
>> >>> AFAIK, the purpose of these modules is to keep related tests/test-utils/utils together. We are not expecting users to make use of such artifacts. Please let me know if you have any objections. If there are none and the PR gets merged, the artifacts will no longer be published.
>> >>>
>> >>> Thanks!
>> >>> Łukasz
Re: Stop publishing unneeded Java artifacts
Posted by Kenneth Knowles <ke...@apache.org>.
I know that the Nemo project had tested the Nexmark pipelines. They may
only need the main code, not the tests themselves. But the tests check that
the runner's output matches a straight-line Java model of the query. So
that is actually a pretty good runner test.
I think any test suite that uses TestPipeline is potentially a useful
integration test jar for runner developed outside our repository.
Kenn
On Tue, Sep 3, 2019 at 10:43 AM Robert Bradshaw <ro...@google.com> wrote:
> :sdks:java:testing:expansion-service could be useful to publish for
> testing as well.
>
> On Fri, Aug 30, 2019 at 3:13 PM Lukasz Cwik <lc...@google.com> wrote:
> >
> > Google internally relies on being able to get the POM files generated
> for:
> > :sdks:java:testing:nexmark
> > :sdks:java:testing:test-utils
> >
> > Generating the POM files currently relies on publishing being enabled
> for those projects so could we keep publishing those two modules. Disabling
> the others sounds fine to me.
> >
> > On Thu, Aug 29, 2019 at 9:41 PM Lukasz Cwik <lc...@google.com> wrote:
> >>
> >> I wanted to double check that we don't rely on this publishing inside
> Google for some reason. Will update this thread tomorrow.
> >>
> >> On Wed, Aug 28, 2019 at 7:11 AM Łukasz Gajowy <lg...@apache.org>
> wrote:
> >>>
> >>> Hi all,
> >>>
> >>> I wanted to notify that in PR 9417 I'm planning to turn off publishing
> of the following modules' artifacts to the maven repository:
> >>>
> >>> :runners:google-cloud-dataflow-java:worker:windmill
> >>> :sdks:java:build-tools
> >>> :sdks:java:javadoc
> >>> :sdks:java:testing:expansion-service
> >>> :sdks:java:io:bigquery-io-perf-tests
> >>> :sdks:java:io:file-based-io-tests
> >>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-2
> >>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-5
> >>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-6
> >>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-common
> >>> :sdks:java:testing:load-tests
> >>> :sdks:java:testing:nexmark
> >>> :sdks:java:testing:test-utils
> >>>
> >>> AFAIK, the purpose of these modules is to keep related
> tests/test-utils/utils together. We are not expecting users to make use of
> such artifacts. Please let me know if you have any objections. If there are
> none and the PR gets merged, the artifacts will no longer be published.
> >>>
> >>> Thanks!
> >>> Łukasz
>
Re: Stop publishing unneeded Java artifacts
Posted by Robert Bradshaw <ro...@google.com>.
:sdks:java:testing:expansion-service could be useful to publish for
testing as well.
On Fri, Aug 30, 2019 at 3:13 PM Lukasz Cwik <lc...@google.com> wrote:
>
> Google internally relies on being able to get the POM files generated for:
> :sdks:java:testing:nexmark
> :sdks:java:testing:test-utils
>
> Generating the POM files currently relies on publishing being enabled for those projects so could we keep publishing those two modules. Disabling the others sounds fine to me.
>
> On Thu, Aug 29, 2019 at 9:41 PM Lukasz Cwik <lc...@google.com> wrote:
>>
>> I wanted to double check that we don't rely on this publishing inside Google for some reason. Will update this thread tomorrow.
>>
>> On Wed, Aug 28, 2019 at 7:11 AM Łukasz Gajowy <lg...@apache.org> wrote:
>>>
>>> Hi all,
>>>
>>> I wanted to notify that in PR 9417 I'm planning to turn off publishing of the following modules' artifacts to the maven repository:
>>>
>>> :runners:google-cloud-dataflow-java:worker:windmill
>>> :sdks:java:build-tools
>>> :sdks:java:javadoc
>>> :sdks:java:testing:expansion-service
>>> :sdks:java:io:bigquery-io-perf-tests
>>> :sdks:java:io:file-based-io-tests
>>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-2
>>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-5
>>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-6
>>> :sdks:java:io:elasticsearch-tests:elasticsearch-tests-common
>>> :sdks:java:testing:load-tests
>>> :sdks:java:testing:nexmark
>>> :sdks:java:testing:test-utils
>>>
>>> AFAIK, the purpose of these modules is to keep related tests/test-utils/utils together. We are not expecting users to make use of such artifacts. Please let me know if you have any objections. If there are none and the PR gets merged, the artifacts will no longer be published.
>>>
>>> Thanks!
>>> Łukasz
Re: Stop publishing unneeded Java artifacts
Posted by Lukasz Cwik <lc...@google.com>.
Google internally relies on being able to get the POM files generated for:
:sdks:java:testing:nexmark
:sdks:java:testing:test-utils
Generating the POM files currently relies on publishing being enabled for
those projects so could we keep publishing those two modules. Disabling the
others sounds fine to me.
On Thu, Aug 29, 2019 at 9:41 PM Lukasz Cwik <lc...@google.com> wrote:
> I wanted to double check that we don't rely on this publishing inside
> Google for some reason. Will update this thread tomorrow.
>
> On Wed, Aug 28, 2019 at 7:11 AM Łukasz Gajowy <lg...@apache.org> wrote:
>
>> Hi all,
>>
>> I wanted to notify that in PR 9417
>> <https://github.com/apache/beam/pull/9417> I'm planning to turn off
>> publishing of the following modules' artifacts to the maven repository:
>>
>> - :runners:google-cloud-dataflow-java:worker:windmill
>> - :sdks:java:build-tools
>> - :sdks:java:javadoc
>> - :sdks:java:testing:expansion-service
>> - :sdks:java:io:bigquery-io-perf-tests
>> - :sdks:java:io:file-based-io-tests
>> - :sdks:java:io:elasticsearch-tests:elasticsearch-tests-2
>> - :sdks:java:io:elasticsearch-tests:elasticsearch-tests-5
>> - :sdks:java:io:elasticsearch-tests:elasticsearch-tests-6
>> - :sdks:java:io:elasticsearch-tests:elasticsearch-tests-common
>> - :sdks:java:testing:load-tests
>> - :sdks:java:testing:nexmark
>> - :sdks:java:testing:test-utils
>>
>> AFAIK, the purpose of these modules is to keep related
>> tests/test-utils/utils together. We are not expecting users to make use of
>> such artifacts. Please let me know if you have any objections. If there are
>> none and the PR gets merged, the artifacts will no longer be published.
>>
>> Thanks!
>> Łukasz
>>
>
Re: Stop publishing unneeded Java artifacts
Posted by Lukasz Cwik <lc...@google.com>.
I wanted to double check that we don't rely on this publishing inside
Google for some reason. Will update this thread tomorrow.
On Wed, Aug 28, 2019 at 7:11 AM Łukasz Gajowy <lg...@apache.org> wrote:
> Hi all,
>
> I wanted to notify that in PR 9417
> <https://github.com/apache/beam/pull/9417> I'm planning to turn off
> publishing of the following modules' artifacts to the maven repository:
>
> - :runners:google-cloud-dataflow-java:worker:windmill
> - :sdks:java:build-tools
> - :sdks:java:javadoc
> - :sdks:java:testing:expansion-service
> - :sdks:java:io:bigquery-io-perf-tests
> - :sdks:java:io:file-based-io-tests
> - :sdks:java:io:elasticsearch-tests:elasticsearch-tests-2
> - :sdks:java:io:elasticsearch-tests:elasticsearch-tests-5
> - :sdks:java:io:elasticsearch-tests:elasticsearch-tests-6
> - :sdks:java:io:elasticsearch-tests:elasticsearch-tests-common
> - :sdks:java:testing:load-tests
> - :sdks:java:testing:nexmark
> - :sdks:java:testing:test-utils
>
> AFAIK, the purpose of these modules is to keep related
> tests/test-utils/utils together. We are not expecting users to make use of
> such artifacts. Please let me know if you have any objections. If there are
> none and the PR gets merged, the artifacts will no longer be published.
>
> Thanks!
> Łukasz
>