You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Luke Cwik <lc...@google.com> on 2021/06/29 19:10:10 UTC
[PROPOSAL] Vendored gRPC 1.36.0 0.2 Release
I noticed that at some point in time we broke support for epoll (and unix
domain sockets) by not relocating and renaming the linux-x86_64 variant.
I filed BEAM-12547[1] and was able to fix the shading to include the
correctly renamed epoll JNI library[2]. I was looking to perform a vendored
gRPC 1.36.0 release and follow-up by fixing our existing test to catch the
breakage by requiring ManagedChannelFactoryTest to run if the architecture
is Linux instead of when Epoll is available[3].
Any concerns on the release?
1: https://issues.apache.org/jira/browse/BEAM-12547
2: https://github.com/apache/beam/pull/15091
3:
https://github.com/apache/beam/blob/5fffad689e3fa7b6f788f3037a02fb215c1864ec/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/channel/ManagedChannelFactoryTest.java#L48
Re: [PROPOSAL] Vendored gRPC 1.36.0 0.2 Release
Posted by Luke Cwik <lc...@google.com>.
Great. Will start the release process now.
On Wed, Jun 30, 2021 at 8:42 AM Tomo Suzuki <su...@google.com> wrote:
> Thanks. I don't have any other items.
>
> On Wed, Jun 30, 2021 at 11:01 AM Ismaël Mejía <ie...@gmail.com> wrote:
>
>> +1 I just merged Tomo's security fix so this should be ready to go
>>
>> On Wed, Jun 30, 2021 at 12:26 AM Luke Cwik <lc...@google.com> wrote:
>> >
>> > Sounds good, will wait for you PR.
>> >
>> > On Tue, Jun 29, 2021 at 2:35 PM Tomo Suzuki <su...@google.com> wrote:
>> >>
>> >> I have this https://issues.apache.org/jira/browse/BEAM-12422
>> "Vendored gRPC 1.36.0 is using a log4j version with security issues".
>> >> Let me try to fix that.
>> >>
>> >> On Tue, Jun 29, 2021 at 3:57 PM Luke Cwik <lc...@google.com> wrote:
>> >>>
>> >>> Yes, but as I suggested we should make the test mandatory if you're
>> using Linux to catch the breakage in the future.
>> >>>
>> >>> On Tue, Jun 29, 2021 at 12:22 PM Kenneth Knowles <ke...@apache.org>
>> wrote:
>> >>>>
>> >>>> Sounds good. I imagine the tests are set up that way so that users
>> can run tests locally on platforms that do not support epoll. Will the
>> change in shaded deps preserve this?
>> >>>>
>> >>>> Kenn
>> >>>>
>> >>>> On Tue, Jun 29, 2021 at 12:10 PM Luke Cwik <lc...@google.com> wrote:
>> >>>>>
>> >>>>> I noticed that at some point in time we broke support for epoll
>> (and unix domain sockets) by not relocating and renaming the linux-x86_64
>> variant.
>> >>>>>
>> >>>>> I filed BEAM-12547[1] and was able to fix the shading to include
>> the correctly renamed epoll JNI library[2]. I was looking to perform a
>> vendored gRPC 1.36.0 release and follow-up by fixing our existing test to
>> catch the breakage by requiring ManagedChannelFactoryTest to run if the
>> architecture is Linux instead of when Epoll is available[3].
>> >>>>>
>> >>>>> Any concerns on the release?
>> >>>>>
>> >>>>> 1: https://issues.apache.org/jira/browse/BEAM-12547
>> >>>>> 2: https://github.com/apache/beam/pull/15091
>> >>>>> 3:
>> https://github.com/apache/beam/blob/5fffad689e3fa7b6f788f3037a02fb215c1864ec/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/channel/ManagedChannelFactoryTest.java#L48
>> >>
>> >>
>> >>
>> >> --
>> >> Regards,
>> >> Tomo
>>
>
>
> --
> Regards,
> Tomo
>
Re: [PROPOSAL] Vendored gRPC 1.36.0 0.2 Release
Posted by Tomo Suzuki <su...@google.com>.
Thanks. I don't have any other items.
On Wed, Jun 30, 2021 at 11:01 AM Ismaël Mejía <ie...@gmail.com> wrote:
> +1 I just merged Tomo's security fix so this should be ready to go
>
> On Wed, Jun 30, 2021 at 12:26 AM Luke Cwik <lc...@google.com> wrote:
> >
> > Sounds good, will wait for you PR.
> >
> > On Tue, Jun 29, 2021 at 2:35 PM Tomo Suzuki <su...@google.com> wrote:
> >>
> >> I have this https://issues.apache.org/jira/browse/BEAM-12422 "Vendored
> gRPC 1.36.0 is using a log4j version with security issues".
> >> Let me try to fix that.
> >>
> >> On Tue, Jun 29, 2021 at 3:57 PM Luke Cwik <lc...@google.com> wrote:
> >>>
> >>> Yes, but as I suggested we should make the test mandatory if you're
> using Linux to catch the breakage in the future.
> >>>
> >>> On Tue, Jun 29, 2021 at 12:22 PM Kenneth Knowles <ke...@apache.org>
> wrote:
> >>>>
> >>>> Sounds good. I imagine the tests are set up that way so that users
> can run tests locally on platforms that do not support epoll. Will the
> change in shaded deps preserve this?
> >>>>
> >>>> Kenn
> >>>>
> >>>> On Tue, Jun 29, 2021 at 12:10 PM Luke Cwik <lc...@google.com> wrote:
> >>>>>
> >>>>> I noticed that at some point in time we broke support for epoll (and
> unix domain sockets) by not relocating and renaming the linux-x86_64
> variant.
> >>>>>
> >>>>> I filed BEAM-12547[1] and was able to fix the shading to include the
> correctly renamed epoll JNI library[2]. I was looking to perform a vendored
> gRPC 1.36.0 release and follow-up by fixing our existing test to catch the
> breakage by requiring ManagedChannelFactoryTest to run if the architecture
> is Linux instead of when Epoll is available[3].
> >>>>>
> >>>>> Any concerns on the release?
> >>>>>
> >>>>> 1: https://issues.apache.org/jira/browse/BEAM-12547
> >>>>> 2: https://github.com/apache/beam/pull/15091
> >>>>> 3:
> https://github.com/apache/beam/blob/5fffad689e3fa7b6f788f3037a02fb215c1864ec/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/channel/ManagedChannelFactoryTest.java#L48
> >>
> >>
> >>
> >> --
> >> Regards,
> >> Tomo
>
--
Regards,
Tomo
Re: [PROPOSAL] Vendored gRPC 1.36.0 0.2 Release
Posted by Ismaël Mejía <ie...@gmail.com>.
+1 I just merged Tomo's security fix so this should be ready to go
On Wed, Jun 30, 2021 at 12:26 AM Luke Cwik <lc...@google.com> wrote:
>
> Sounds good, will wait for you PR.
>
> On Tue, Jun 29, 2021 at 2:35 PM Tomo Suzuki <su...@google.com> wrote:
>>
>> I have this https://issues.apache.org/jira/browse/BEAM-12422 "Vendored gRPC 1.36.0 is using a log4j version with security issues".
>> Let me try to fix that.
>>
>> On Tue, Jun 29, 2021 at 3:57 PM Luke Cwik <lc...@google.com> wrote:
>>>
>>> Yes, but as I suggested we should make the test mandatory if you're using Linux to catch the breakage in the future.
>>>
>>> On Tue, Jun 29, 2021 at 12:22 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>>
>>>> Sounds good. I imagine the tests are set up that way so that users can run tests locally on platforms that do not support epoll. Will the change in shaded deps preserve this?
>>>>
>>>> Kenn
>>>>
>>>> On Tue, Jun 29, 2021 at 12:10 PM Luke Cwik <lc...@google.com> wrote:
>>>>>
>>>>> I noticed that at some point in time we broke support for epoll (and unix domain sockets) by not relocating and renaming the linux-x86_64 variant.
>>>>>
>>>>> I filed BEAM-12547[1] and was able to fix the shading to include the correctly renamed epoll JNI library[2]. I was looking to perform a vendored gRPC 1.36.0 release and follow-up by fixing our existing test to catch the breakage by requiring ManagedChannelFactoryTest to run if the architecture is Linux instead of when Epoll is available[3].
>>>>>
>>>>> Any concerns on the release?
>>>>>
>>>>> 1: https://issues.apache.org/jira/browse/BEAM-12547
>>>>> 2: https://github.com/apache/beam/pull/15091
>>>>> 3: https://github.com/apache/beam/blob/5fffad689e3fa7b6f788f3037a02fb215c1864ec/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/channel/ManagedChannelFactoryTest.java#L48
>>
>>
>>
>> --
>> Regards,
>> Tomo
Re: [PROPOSAL] Vendored gRPC 1.36.0 0.2 Release
Posted by Luke Cwik <lc...@google.com>.
Sounds good, will wait for you PR.
On Tue, Jun 29, 2021 at 2:35 PM Tomo Suzuki <su...@google.com> wrote:
> I have this https://issues.apache.org/jira/browse/BEAM-12422 "Vendored
> gRPC 1.36.0 is using a log4j version with security issues".
> Let me try to fix that.
>
> On Tue, Jun 29, 2021 at 3:57 PM Luke Cwik <lc...@google.com> wrote:
>
>> Yes, but as I suggested we should make the test mandatory if you're using
>> Linux to catch the breakage in the future.
>>
>> On Tue, Jun 29, 2021 at 12:22 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> Sounds good. I imagine the tests are set up that way so that users can
>>> run tests locally on platforms that do not support epoll. Will the change
>>> in shaded deps preserve this?
>>>
>>> Kenn
>>>
>>> On Tue, Jun 29, 2021 at 12:10 PM Luke Cwik <lc...@google.com> wrote:
>>>
>>>> I noticed that at some point in time we broke support for epoll (and
>>>> unix domain sockets) by not relocating and renaming the linux-x86_64
>>>> variant.
>>>>
>>>> I filed BEAM-12547[1] and was able to fix the shading to include the
>>>> correctly renamed epoll JNI library[2]. I was looking to perform a vendored
>>>> gRPC 1.36.0 release and follow-up by fixing our existing test to catch the
>>>> breakage by requiring ManagedChannelFactoryTest to run if the architecture
>>>> is Linux instead of when Epoll is available[3].
>>>>
>>>> Any concerns on the release?
>>>>
>>>> 1: https://issues.apache.org/jira/browse/BEAM-12547
>>>> 2: https://github.com/apache/beam/pull/15091
>>>> 3:
>>>> https://github.com/apache/beam/blob/5fffad689e3fa7b6f788f3037a02fb215c1864ec/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/channel/ManagedChannelFactoryTest.java#L48
>>>>
>>>
>
> --
> Regards,
> Tomo
>
Re: [PROPOSAL] Vendored gRPC 1.36.0 0.2 Release
Posted by Tomo Suzuki <su...@google.com>.
I have this https://issues.apache.org/jira/browse/BEAM-12422 "Vendored gRPC
1.36.0 is using a log4j version with security issues".
Let me try to fix that.
On Tue, Jun 29, 2021 at 3:57 PM Luke Cwik <lc...@google.com> wrote:
> Yes, but as I suggested we should make the test mandatory if you're using
> Linux to catch the breakage in the future.
>
> On Tue, Jun 29, 2021 at 12:22 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> Sounds good. I imagine the tests are set up that way so that users can
>> run tests locally on platforms that do not support epoll. Will the change
>> in shaded deps preserve this?
>>
>> Kenn
>>
>> On Tue, Jun 29, 2021 at 12:10 PM Luke Cwik <lc...@google.com> wrote:
>>
>>> I noticed that at some point in time we broke support for epoll (and
>>> unix domain sockets) by not relocating and renaming the linux-x86_64
>>> variant.
>>>
>>> I filed BEAM-12547[1] and was able to fix the shading to include the
>>> correctly renamed epoll JNI library[2]. I was looking to perform a vendored
>>> gRPC 1.36.0 release and follow-up by fixing our existing test to catch the
>>> breakage by requiring ManagedChannelFactoryTest to run if the architecture
>>> is Linux instead of when Epoll is available[3].
>>>
>>> Any concerns on the release?
>>>
>>> 1: https://issues.apache.org/jira/browse/BEAM-12547
>>> 2: https://github.com/apache/beam/pull/15091
>>> 3:
>>> https://github.com/apache/beam/blob/5fffad689e3fa7b6f788f3037a02fb215c1864ec/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/channel/ManagedChannelFactoryTest.java#L48
>>>
>>
--
Regards,
Tomo
Re: [PROPOSAL] Vendored gRPC 1.36.0 0.2 Release
Posted by Luke Cwik <lc...@google.com>.
Yes, but as I suggested we should make the test mandatory if you're using
Linux to catch the breakage in the future.
On Tue, Jun 29, 2021 at 12:22 PM Kenneth Knowles <ke...@apache.org> wrote:
> Sounds good. I imagine the tests are set up that way so that users can run
> tests locally on platforms that do not support epoll. Will the change in
> shaded deps preserve this?
>
> Kenn
>
> On Tue, Jun 29, 2021 at 12:10 PM Luke Cwik <lc...@google.com> wrote:
>
>> I noticed that at some point in time we broke support for epoll (and unix
>> domain sockets) by not relocating and renaming the linux-x86_64 variant.
>>
>> I filed BEAM-12547[1] and was able to fix the shading to include the
>> correctly renamed epoll JNI library[2]. I was looking to perform a vendored
>> gRPC 1.36.0 release and follow-up by fixing our existing test to catch the
>> breakage by requiring ManagedChannelFactoryTest to run if the architecture
>> is Linux instead of when Epoll is available[3].
>>
>> Any concerns on the release?
>>
>> 1: https://issues.apache.org/jira/browse/BEAM-12547
>> 2: https://github.com/apache/beam/pull/15091
>> 3:
>> https://github.com/apache/beam/blob/5fffad689e3fa7b6f788f3037a02fb215c1864ec/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/channel/ManagedChannelFactoryTest.java#L48
>>
>
Re: [PROPOSAL] Vendored gRPC 1.36.0 0.2 Release
Posted by Kenneth Knowles <ke...@apache.org>.
Sounds good. I imagine the tests are set up that way so that users can run
tests locally on platforms that do not support epoll. Will the change in
shaded deps preserve this?
Kenn
On Tue, Jun 29, 2021 at 12:10 PM Luke Cwik <lc...@google.com> wrote:
> I noticed that at some point in time we broke support for epoll (and unix
> domain sockets) by not relocating and renaming the linux-x86_64 variant.
>
> I filed BEAM-12547[1] and was able to fix the shading to include the
> correctly renamed epoll JNI library[2]. I was looking to perform a vendored
> gRPC 1.36.0 release and follow-up by fixing our existing test to catch the
> breakage by requiring ManagedChannelFactoryTest to run if the architecture
> is Linux instead of when Epoll is available[3].
>
> Any concerns on the release?
>
> 1: https://issues.apache.org/jira/browse/BEAM-12547
> 2: https://github.com/apache/beam/pull/15091
> 3:
> https://github.com/apache/beam/blob/5fffad689e3fa7b6f788f3037a02fb215c1864ec/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/channel/ManagedChannelFactoryTest.java#L48
>