You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Scott Wegner <sc...@apache.org> on 2019/01/02 17:19:20 UTC
Re: Beam snapshots broken
Yes, I believe this will be fixed with the vendoring release.
I am back from the holidays now and ready to pick this up.
On Thu, Dec 27, 2018 at 10:56 AM Ismaël Mejía <ie...@gmail.com> wrote:
> Thanks Andrew for the info.
> I think things can wait a bit considering the time of the year, I just
> wanted to raise awareness about the issue.
> I suppose that we can wait, it should not be long before the vendoring
> release is done and this will fix this if I understood correctly.
> If anyone else is blocked on this please contact and we will revert
> it, otherwise I suppose we can do this revert locally (as Gleb
> mentioned) in the meantime.
>
> On Thu, Dec 27, 2018 at 6:24 PM Andrew Pilloud <ap...@google.com>
> wrote:
> >
> > https://issues.apache.org/jira/browse/BEAM-6282
> >
> > Kenn (and everyone else who has context on this change) are out this
> week, so I don't think anyone is making progress on it. Is this something
> that can wait a week or two? If not we should revert
> https://github.com/apache/beam/pull/7324
> >
> > Andrew
> >
> > On Thu, Dec 27, 2018 at 9:07 AM Gleb Kanterov <gl...@spotify.com> wrote:
> >>
> >> I can reproduce this on my machine, and reverting
> https://github.com/apache/beam/pull/7324 fixed the problem. There is a
> separate thread in dev@ about releasing vendored gRPC v0.2, I'm wondering
> if it will this issue.
> >>
> >> On Thu, Dec 27, 2018 at 5:20 PM Ismaël Mejía <ie...@gmail.com> wrote:
> >>>
> >>> Looks like snapshots are broken again since 20 december, can somebody
> PTAL?
> >>> Seems like some part of the vendoring could be related to this failure
> >>> (maybe it is looking for the unpublished version)?
> >>>
> >>> Running some tests in one existing application I found this
> >>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
> >>> elapsed: 0.447 s <<< FAILURE! - in SerializationTest
> >>> [ERROR] nonSerilizableTest(SerializationTest) Time elapsed: 0.028 s
> <<< ERROR!
> >>> java.lang.NoClassDefFoundError:
> >>>
> org/apache/beam/vendor/grpc/v1p13p1/com/google/protobuf/ProtocolMessageEnum
> >>> at SerializationTest.nonSerilizableTest(SerializationTest.java:27)
> >>> Caused by: java.lang.ClassNotFoundException:
> >>>
> org.apache.beam.vendor.grpc.v1p13p1.com.google.protobuf.ProtocolMessageEnum
> >>> at SerializationTest.nonSerilizableTest(SerializationTest.java:27)
> >>>
> >>> On Thu, Dec 13, 2018 at 9:13 AM Mark Liu <ma...@google.com> wrote:
> >>> >
> >>> > Looks like the recent failure (like this job) is related to
> ':beam-sdks-python:test' change introduced in this PR. `./gradlew
> :beam-sdks-python:test` can reproduce the error.
> >>> >
> >>> > Testing a fix in PR7273.
> >>> >
> >>> > On Wed, Dec 12, 2018 at 8:31 AM Yifan Zou <yi...@google.com>
> wrote:
> >>> >>
> >>> >> Beam9 is offline right now. But, the job also failed on beam4 and
> 13 with "Could not determine the dependencies of task
> ':beam-sdks-python:test.".
> >>> >> Seems like the task dependency did not setup properly.
> >>> >>
> >>> >>
> >>> >>
> >>> >> On Wed, Dec 12, 2018 at 2:03 AM Ismaël Mejía <ie...@gmail.com>
> wrote:
> >>> >>>
> >>> >>> You are right it seems that it was related to beam9 (wondering if
> it
> >>> >>> was bad luck that it was always assigned to beam9 or we can improve
> >>> >>> that poor balancing error).
> >>> >>> However it failed again today against beam13 maybe this time is
> just a
> >>> >>> build issue but seems related to python too.
> >>> >>>
> >>> >>> On Tue, Dec 11, 2018 at 7:33 PM Boyuan Zhang <bo...@google.com>
> wrote:
> >>> >>> >
> >>> >>> > Seems like, all failed jobs are not owing to the single task
> failure. There failed task were executed on beam9, which was rebooted
> yesterday because python tests failed continuously. +Yifan Zou may have
> more useful content here.
> >>> >>> >
> >>> >>> > On Tue, Dec 11, 2018 at 9:10 AM Ismaël Mejía <ie...@gmail.com>
> wrote:
> >>> >>> >>
> >>> >>> >> It seems that Beam snapshots are broken since Dec. 2
> >>> >>> >>
> https://builds.apache.org/view/A-D/view/Beam/job/beam_Release_Gradle_NightlySnapshot/
> >>> >>> >>
> >>> >>> >> It seems "The :beam-website:startDockerContainer task failed."
> >>> >>> >> Can somebody please take a look.
> >>
> >>
> >>
> >> --
> >> Cheers,
> >> Gleb
>
--
Got feedback? tinyurl.com/swegner-feedback
Re: Beam snapshots broken
Posted by Alexey Romanenko <ar...@gmail.com>.
Thank you for fixing this, Scott. I also can confirm that it works for me now.
> On 2 Jan 2019, at 22:36, Ismaël Mejía <ie...@gmail.com> wrote:
>
> Thanks Scott, I just tested and the errors I found are fixed and
> everything seems to work now. Snapshots are being correctly updated
> too.
>
> On Wed, Jan 2, 2019 at 6:19 PM Scott Wegner <sc...@apache.org> wrote:
>>
>> Yes, I believe this will be fixed with the vendoring release.
>>
>> I am back from the holidays now and ready to pick this up.
>>
>> On Thu, Dec 27, 2018 at 10:56 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>
>>> Thanks Andrew for the info.
>>> I think things can wait a bit considering the time of the year, I just
>>> wanted to raise awareness about the issue.
>>> I suppose that we can wait, it should not be long before the vendoring
>>> release is done and this will fix this if I understood correctly.
>>> If anyone else is blocked on this please contact and we will revert
>>> it, otherwise I suppose we can do this revert locally (as Gleb
>>> mentioned) in the meantime.
>>>
>>> On Thu, Dec 27, 2018 at 6:24 PM Andrew Pilloud <ap...@google.com> wrote:
>>>>
>>>> https://issues.apache.org/jira/browse/BEAM-6282
>>>>
>>>> Kenn (and everyone else who has context on this change) are out this week, so I don't think anyone is making progress on it. Is this something that can wait a week or two? If not we should revert https://github.com/apache/beam/pull/7324
>>>>
>>>> Andrew
>>>>
>>>> On Thu, Dec 27, 2018 at 9:07 AM Gleb Kanterov <gl...@spotify.com> wrote:
>>>>>
>>>>> I can reproduce this on my machine, and reverting https://github.com/apache/beam/pull/7324 fixed the problem. There is a separate thread in dev@ about releasing vendored gRPC v0.2, I'm wondering if it will this issue.
>>>>>
>>>>> On Thu, Dec 27, 2018 at 5:20 PM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>>>
>>>>>> Looks like snapshots are broken again since 20 december, can somebody PTAL?
>>>>>> Seems like some part of the vendoring could be related to this failure
>>>>>> (maybe it is looking for the unpublished version)?
>>>>>>
>>>>>> Running some tests in one existing application I found this
>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
>>>>>> elapsed: 0.447 s <<< FAILURE! - in SerializationTest
>>>>>> [ERROR] nonSerilizableTest(SerializationTest) Time elapsed: 0.028 s <<< ERROR!
>>>>>> java.lang.NoClassDefFoundError:
>>>>>> org/apache/beam/vendor/grpc/v1p13p1/com/google/protobuf/ProtocolMessageEnum
>>>>>> at SerializationTest.nonSerilizableTest(SerializationTest.java:27)
>>>>>> Caused by: java.lang.ClassNotFoundException:
>>>>>> org.apache.beam.vendor.grpc.v1p13p1.com.google.protobuf.ProtocolMessageEnum
>>>>>> at SerializationTest.nonSerilizableTest(SerializationTest.java:27)
>>>>>>
>>>>>> On Thu, Dec 13, 2018 at 9:13 AM Mark Liu <ma...@google.com> wrote:
>>>>>>>
>>>>>>> Looks like the recent failure (like this job) is related to ':beam-sdks-python:test' change introduced in this PR. `./gradlew :beam-sdks-python:test` can reproduce the error.
>>>>>>>
>>>>>>> Testing a fix in PR7273.
>>>>>>>
>>>>>>> On Wed, Dec 12, 2018 at 8:31 AM Yifan Zou <yi...@google.com> wrote:
>>>>>>>>
>>>>>>>> Beam9 is offline right now. But, the job also failed on beam4 and 13 with "Could not determine the dependencies of task ':beam-sdks-python:test.".
>>>>>>>> Seems like the task dependency did not setup properly.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Dec 12, 2018 at 2:03 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> You are right it seems that it was related to beam9 (wondering if it
>>>>>>>>> was bad luck that it was always assigned to beam9 or we can improve
>>>>>>>>> that poor balancing error).
>>>>>>>>> However it failed again today against beam13 maybe this time is just a
>>>>>>>>> build issue but seems related to python too.
>>>>>>>>>
>>>>>>>>> On Tue, Dec 11, 2018 at 7:33 PM Boyuan Zhang <bo...@google.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Seems like, all failed jobs are not owing to the single task failure. There failed task were executed on beam9, which was rebooted yesterday because python tests failed continuously. +Yifan Zou may have more useful content here.
>>>>>>>>>>
>>>>>>>>>> On Tue, Dec 11, 2018 at 9:10 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> It seems that Beam snapshots are broken since Dec. 2
>>>>>>>>>>> https://builds.apache.org/view/A-D/view/Beam/job/beam_Release_Gradle_NightlySnapshot/
>>>>>>>>>>>
>>>>>>>>>>> It seems "The :beam-website:startDockerContainer task failed."
>>>>>>>>>>> Can somebody please take a look.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Cheers,
>>>>> Gleb
>>
>>
>>
>> --
>>
>>
>>
>>
>> Got feedback? tinyurl.com/swegner-feedback
Re: Beam snapshots broken
Posted by Ismaël Mejía <ie...@gmail.com>.
Thanks Scott, I just tested and the errors I found are fixed and
everything seems to work now. Snapshots are being correctly updated
too.
On Wed, Jan 2, 2019 at 6:19 PM Scott Wegner <sc...@apache.org> wrote:
>
> Yes, I believe this will be fixed with the vendoring release.
>
> I am back from the holidays now and ready to pick this up.
>
> On Thu, Dec 27, 2018 at 10:56 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>
>> Thanks Andrew for the info.
>> I think things can wait a bit considering the time of the year, I just
>> wanted to raise awareness about the issue.
>> I suppose that we can wait, it should not be long before the vendoring
>> release is done and this will fix this if I understood correctly.
>> If anyone else is blocked on this please contact and we will revert
>> it, otherwise I suppose we can do this revert locally (as Gleb
>> mentioned) in the meantime.
>>
>> On Thu, Dec 27, 2018 at 6:24 PM Andrew Pilloud <ap...@google.com> wrote:
>> >
>> > https://issues.apache.org/jira/browse/BEAM-6282
>> >
>> > Kenn (and everyone else who has context on this change) are out this week, so I don't think anyone is making progress on it. Is this something that can wait a week or two? If not we should revert https://github.com/apache/beam/pull/7324
>> >
>> > Andrew
>> >
>> > On Thu, Dec 27, 2018 at 9:07 AM Gleb Kanterov <gl...@spotify.com> wrote:
>> >>
>> >> I can reproduce this on my machine, and reverting https://github.com/apache/beam/pull/7324 fixed the problem. There is a separate thread in dev@ about releasing vendored gRPC v0.2, I'm wondering if it will this issue.
>> >>
>> >> On Thu, Dec 27, 2018 at 5:20 PM Ismaël Mejía <ie...@gmail.com> wrote:
>> >>>
>> >>> Looks like snapshots are broken again since 20 december, can somebody PTAL?
>> >>> Seems like some part of the vendoring could be related to this failure
>> >>> (maybe it is looking for the unpublished version)?
>> >>>
>> >>> Running some tests in one existing application I found this
>> >>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
>> >>> elapsed: 0.447 s <<< FAILURE! - in SerializationTest
>> >>> [ERROR] nonSerilizableTest(SerializationTest) Time elapsed: 0.028 s <<< ERROR!
>> >>> java.lang.NoClassDefFoundError:
>> >>> org/apache/beam/vendor/grpc/v1p13p1/com/google/protobuf/ProtocolMessageEnum
>> >>> at SerializationTest.nonSerilizableTest(SerializationTest.java:27)
>> >>> Caused by: java.lang.ClassNotFoundException:
>> >>> org.apache.beam.vendor.grpc.v1p13p1.com.google.protobuf.ProtocolMessageEnum
>> >>> at SerializationTest.nonSerilizableTest(SerializationTest.java:27)
>> >>>
>> >>> On Thu, Dec 13, 2018 at 9:13 AM Mark Liu <ma...@google.com> wrote:
>> >>> >
>> >>> > Looks like the recent failure (like this job) is related to ':beam-sdks-python:test' change introduced in this PR. `./gradlew :beam-sdks-python:test` can reproduce the error.
>> >>> >
>> >>> > Testing a fix in PR7273.
>> >>> >
>> >>> > On Wed, Dec 12, 2018 at 8:31 AM Yifan Zou <yi...@google.com> wrote:
>> >>> >>
>> >>> >> Beam9 is offline right now. But, the job also failed on beam4 and 13 with "Could not determine the dependencies of task ':beam-sdks-python:test.".
>> >>> >> Seems like the task dependency did not setup properly.
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> On Wed, Dec 12, 2018 at 2:03 AM Ismaël Mejía <ie...@gmail.com> wrote:
>> >>> >>>
>> >>> >>> You are right it seems that it was related to beam9 (wondering if it
>> >>> >>> was bad luck that it was always assigned to beam9 or we can improve
>> >>> >>> that poor balancing error).
>> >>> >>> However it failed again today against beam13 maybe this time is just a
>> >>> >>> build issue but seems related to python too.
>> >>> >>>
>> >>> >>> On Tue, Dec 11, 2018 at 7:33 PM Boyuan Zhang <bo...@google.com> wrote:
>> >>> >>> >
>> >>> >>> > Seems like, all failed jobs are not owing to the single task failure. There failed task were executed on beam9, which was rebooted yesterday because python tests failed continuously. +Yifan Zou may have more useful content here.
>> >>> >>> >
>> >>> >>> > On Tue, Dec 11, 2018 at 9:10 AM Ismaël Mejía <ie...@gmail.com> wrote:
>> >>> >>> >>
>> >>> >>> >> It seems that Beam snapshots are broken since Dec. 2
>> >>> >>> >> https://builds.apache.org/view/A-D/view/Beam/job/beam_Release_Gradle_NightlySnapshot/
>> >>> >>> >>
>> >>> >>> >> It seems "The :beam-website:startDockerContainer task failed."
>> >>> >>> >> Can somebody please take a look.
>> >>
>> >>
>> >>
>> >> --
>> >> Cheers,
>> >> Gleb
>
>
>
> --
>
>
>
>
> Got feedback? tinyurl.com/swegner-feedback