You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Alexey Romanenko <ar...@gmail.com> on 2022/05/31 14:24:12 UTC
Failing beam_SeedJob
Hi everyone,
Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The last successful build [2] was 12 days ago.
Does anyone know the reasons of this?
—
Alexey
[1] https://ci-beam.apache.org/job/beam_SeedJob/ <https://ci-beam.apache.org/job/beam_SeedJob/>
[2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/ <https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/>
Re: Failing beam_SeedJob
Posted by Valentyn Tymofieiev <va...@google.com>.
On Fri, Jun 3, 2022 at 5:30 PM Alexey Romanenko <ar...@gmail.com>
wrote:
> Thanks Valentyn, it may help!
>
> I’m wondering if the seed job was broken because of Pablo’s change or it
> just happened the same issue like in this thread and it was fixed just by
> Jenkins restart.
>
Good question... Maybe it was not related... since immediately after we
reverted Pablo's changes, Jenkins was still broken. +Pablo Estrada
<pa...@google.com> do you want to give
https://github.com/apache/beam/pull/17468 another try?
>
> On 3 Jun 2022, at 15:22, Valentyn Tymofieiev <va...@google.com> wrote:
>
> I am not aware of a straightforward way. I have been using this script:
> https://gist.github.com/tvalentyn/08cdb2836d311c55ae6ec09a276aea2a
> +Pablo Estrada <pa...@google.com> investigated this in
> https://issues.apache.org/jira/browse/BEAM-14377 but Pablo's changes were
> rolled back because somehow seed job became broken.
>
> On Fri, Jun 3, 2022 at 2:56 PM Alexey Romanenko <ar...@gmail.com>
> wrote:
>
>> Yes, it’s ok now.
>> Thanks all!
>>
>> Side question - is there a way to run a Jenkins job against specific PR?
>> I tried to specify different git refs as an input parameter but it didn’t
>> work for me…
>>
>> —
>> Alexey
>>
>> On 3 Jun 2022, at 05:02, Ahmet Altay <al...@google.com> wrote:
>>
>> Seed job looks healthy again:
>> https://ci-beam.apache.org/job/beam_SeedJob/9762/ .
>>
>> On Thu, Jun 2, 2022 at 10:24 AM Ryan Thompson <ry...@google.com>
>> wrote:
>>
>>> I've filed a jira to infra and asked on the slack channel. Hopefully
>>> someone there has the power to restart the server.
>>>
>>> https://issues.apache.org/jira/browse/INFRA-23335
>>>
>>> On Thu, Jun 2, 2022 at 12:41 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>
>>>> Commented on the Jira. Seems like it happens somewhat rarely, but is
>>>> sometimes resolved by a restart, and sometimes has to do with some version
>>>> mismatch issues. I did not find anything like a real root cause, just
>>>> trial-and-error fixes. I'm not certain what may have occurred around May 19
>>>> to this infrastructure. Hoping INFRA can help us sort it out.
>>>>
>>>> Kenn
>>>>
>>>> On Thu, Jun 2, 2022 at 9:15 AM Alexey Romanenko <
>>>> aromanenko.dev@gmail.com> wrote:
>>>>
>>>>> Thanks Ahmet and Ryan for taking a look!
>>>>>
>>>>> I agree that referenced commits seems are not related, that us why I
>>>>> was puzzled with this.
>>>>>
>>>>> On 2 Jun 2022, at 17:56, Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jun 2, 2022 at 8:53 AM Ryan Thompson <ry...@google.com>
>>>>> wrote:
>>>>>
>>>>>> I asked in the slack channel to restart jenkins. I'm looking through
>>>>>> the past messages to see if there's someone there I can tag.
>>>>>>
>>>>>
>>>>> Thank you. Unfortunately, I do not believe we have a single expert we
>>>>> can tag.
>>>>>
>>>>>
>>>>>> Am I right in understanding this service is managed by someone in
>>>>>> apache and not us?
>>>>>>
>>>>>
>>>>> It is mixed. ASF Infra works with a vendor and they run a hosted
>>>>> jenkins instance. We control the configuration of the instance, and the
>>>>> worker nodes.
>>>>>
>>>>>
>>>>>>
>>>>>> On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>> /cc @Kenneth Knowles <kl...@google.com>
>>>>>>>
>>>>>>> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <al...@google.com> wrote:
>>>>>>>
>>>>>>>> I do not have a great idea but googling about the error, similar
>>>>>>>> errors were resolved by restarting jenkins. We could try that. We may need
>>>>>>>> to ask infra. ( @Ryan Thompson <ry...@google.com> - could
>>>>>>>> you please infra to restart jenkins?)
>>>>>>>>
>>>>>>>> I do not think this issue is related to any change in the source
>>>>>>>> control. The commit referenced in the first failed job is an unrelated doc
>>>>>>>> change.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <
>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I created a jira for this (not sure if it’s P0, but P1 for sure):
>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-14548
>>>>>>>>>
>>>>>>>>> Could someone, who has more knowledge than me in Beam
>>>>>>>>> infrastructure/Jenkins, take a look, please?
>>>>>>>>>
>>>>>>>>> —
>>>>>>>>> Alexey
>>>>>>>>>
>>>>>>>>> On 31 May 2022, at 17:08, Alexey Romanenko <
>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> The first failed job is
>>>>>>>>> https://ci-beam.apache.org/job/beam_SeedJob/9696/
>>>>>>>>> It fails with this error (that says not so much):
>>>>>>>>>
>>>>>>>>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>>>>>>>>> Processing DSL script
>>>>>>>>> .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>>>>>>>>> Processing DSL script
>>>>>>>>> .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>>>>>>>>> Processing DSL script
>>>>>>>>> .test-infra/jenkins/job_Dependency_Check.groovy
>>>>>>>>> ERROR: java.io.IOException: Failed to persist config.xml
>>>>>>>>>
>>>>>>>>> and I don’t see any recent changes for these files.
>>>>>>>>>
>>>>>>>>> On 31 May 2022, at 16:24, Alexey Romanenko <
>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Hi everyone,
>>>>>>>>>
>>>>>>>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th.
>>>>>>>>> The last successful build [2] was 12 days ago.
>>>>>>>>> Does anyone know the reasons of this?
>>>>>>>>>
>>>>>>>>> —
>>>>>>>>> Alexey
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/
>>>>>>>>> [2]
>>>>>>>>> https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>
>>
>
Re: Failing beam_SeedJob
Posted by Pablo Estrada <pa...@google.com>.
Hi all,
for what it's worth, I have been able to run Seed Jobs from the UI by doing
this change[1] by hand, without causing breakage.
1) Open Jenkins SeedJob page
2) Change its configuration to add the ghprbPullId parameter and save
3) Run a Seed Job by passing the build parameter and the ghprbPullId
parameter.
Eventually the seed job configuration ran again, and removed
the ghprbPullId parameter, but I've done this to run the seed job in PRs
before.
[1]
https://github.com/apache/beam/pull/17468/files#diff-b5a62b477e12af6cdb39b340db42e06c017cc06f7b595e2bfed5a5d935c46972R66-R69
On Fri, Jun 3, 2022 at 8:30 AM Alexey Romanenko <ar...@gmail.com>
wrote:
> Thanks Valentyn, it may help!
>
> I’m wondering if the seed job was broken because of Pablo’s change or it
> just happened the same issue like in this thread and it was fixed just by
> Jenkins restart.
>
> On 3 Jun 2022, at 15:22, Valentyn Tymofieiev <va...@google.com> wrote:
>
> I am not aware of a straightforward way. I have been using this script:
> https://gist.github.com/tvalentyn/08cdb2836d311c55ae6ec09a276aea2a
> +Pablo Estrada <pa...@google.com> investigated this in
> https://issues.apache.org/jira/browse/BEAM-14377 but Pablo's changes were
> rolled back because somehow seed job became broken.
>
> On Fri, Jun 3, 2022 at 2:56 PM Alexey Romanenko <ar...@gmail.com>
> wrote:
>
>> Yes, it’s ok now.
>> Thanks all!
>>
>> Side question - is there a way to run a Jenkins job against specific PR?
>> I tried to specify different git refs as an input parameter but it didn’t
>> work for me…
>>
>> —
>> Alexey
>>
>> On 3 Jun 2022, at 05:02, Ahmet Altay <al...@google.com> wrote:
>>
>> Seed job looks healthy again:
>> https://ci-beam.apache.org/job/beam_SeedJob/9762/ .
>>
>> On Thu, Jun 2, 2022 at 10:24 AM Ryan Thompson <ry...@google.com>
>> wrote:
>>
>>> I've filed a jira to infra and asked on the slack channel. Hopefully
>>> someone there has the power to restart the server.
>>>
>>> https://issues.apache.org/jira/browse/INFRA-23335
>>>
>>> On Thu, Jun 2, 2022 at 12:41 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>
>>>> Commented on the Jira. Seems like it happens somewhat rarely, but is
>>>> sometimes resolved by a restart, and sometimes has to do with some version
>>>> mismatch issues. I did not find anything like a real root cause, just
>>>> trial-and-error fixes. I'm not certain what may have occurred around May 19
>>>> to this infrastructure. Hoping INFRA can help us sort it out.
>>>>
>>>> Kenn
>>>>
>>>> On Thu, Jun 2, 2022 at 9:15 AM Alexey Romanenko <
>>>> aromanenko.dev@gmail.com> wrote:
>>>>
>>>>> Thanks Ahmet and Ryan for taking a look!
>>>>>
>>>>> I agree that referenced commits seems are not related, that us why I
>>>>> was puzzled with this.
>>>>>
>>>>> On 2 Jun 2022, at 17:56, Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jun 2, 2022 at 8:53 AM Ryan Thompson <ry...@google.com>
>>>>> wrote:
>>>>>
>>>>>> I asked in the slack channel to restart jenkins. I'm looking through
>>>>>> the past messages to see if there's someone there I can tag.
>>>>>>
>>>>>
>>>>> Thank you. Unfortunately, I do not believe we have a single expert we
>>>>> can tag.
>>>>>
>>>>>
>>>>>> Am I right in understanding this service is managed by someone in
>>>>>> apache and not us?
>>>>>>
>>>>>
>>>>> It is mixed. ASF Infra works with a vendor and they run a hosted
>>>>> jenkins instance. We control the configuration of the instance, and the
>>>>> worker nodes.
>>>>>
>>>>>
>>>>>>
>>>>>> On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>> /cc @Kenneth Knowles <kl...@google.com>
>>>>>>>
>>>>>>> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <al...@google.com> wrote:
>>>>>>>
>>>>>>>> I do not have a great idea but googling about the error, similar
>>>>>>>> errors were resolved by restarting jenkins. We could try that. We may need
>>>>>>>> to ask infra. ( @Ryan Thompson <ry...@google.com> - could
>>>>>>>> you please infra to restart jenkins?)
>>>>>>>>
>>>>>>>> I do not think this issue is related to any change in the source
>>>>>>>> control. The commit referenced in the first failed job is an unrelated doc
>>>>>>>> change.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <
>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I created a jira for this (not sure if it’s P0, but P1 for sure):
>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-14548
>>>>>>>>>
>>>>>>>>> Could someone, who has more knowledge than me in Beam
>>>>>>>>> infrastructure/Jenkins, take a look, please?
>>>>>>>>>
>>>>>>>>> —
>>>>>>>>> Alexey
>>>>>>>>>
>>>>>>>>> On 31 May 2022, at 17:08, Alexey Romanenko <
>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> The first failed job is
>>>>>>>>> https://ci-beam.apache.org/job/beam_SeedJob/9696/
>>>>>>>>> It fails with this error (that says not so much):
>>>>>>>>>
>>>>>>>>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>>>>>>>>> Processing DSL script
>>>>>>>>> .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>>>>>>>>> Processing DSL script
>>>>>>>>> .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>>>>>>>>> Processing DSL script
>>>>>>>>> .test-infra/jenkins/job_Dependency_Check.groovy
>>>>>>>>> ERROR: java.io.IOException: Failed to persist config.xml
>>>>>>>>>
>>>>>>>>> and I don’t see any recent changes for these files.
>>>>>>>>>
>>>>>>>>> On 31 May 2022, at 16:24, Alexey Romanenko <
>>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Hi everyone,
>>>>>>>>>
>>>>>>>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th.
>>>>>>>>> The last successful build [2] was 12 days ago.
>>>>>>>>> Does anyone know the reasons of this?
>>>>>>>>>
>>>>>>>>> —
>>>>>>>>> Alexey
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/
>>>>>>>>> [2]
>>>>>>>>> https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>
>>
>
Re: Failing beam_SeedJob
Posted by Alexey Romanenko <ar...@gmail.com>.
Thanks Valentyn, it may help!
I’m wondering if the seed job was broken because of Pablo’s change or it just happened the same issue like in this thread and it was fixed just by Jenkins restart.
> On 3 Jun 2022, at 15:22, Valentyn Tymofieiev <va...@google.com> wrote:
>
> I am not aware of a straightforward way. I have been using this script: https://gist.github.com/tvalentyn/08cdb2836d311c55ae6ec09a276aea2a <https://gist.github.com/tvalentyn/08cdb2836d311c55ae6ec09a276aea2a>
> +Pablo Estrada <ma...@google.com> investigated this in https://issues.apache.org/jira/browse/BEAM-14377 <https://issues.apache.org/jira/browse/BEAM-14377> but Pablo's changes were rolled back because somehow seed job became broken.
>
> On Fri, Jun 3, 2022 at 2:56 PM Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
> Yes, it’s ok now.
> Thanks all!
>
> Side question - is there a way to run a Jenkins job against specific PR? I tried to specify different git refs as an input parameter but it didn’t work for me…
>
> —
> Alexey
>
>> On 3 Jun 2022, at 05:02, Ahmet Altay <altay@google.com <ma...@google.com>> wrote:
>>
>> Seed job looks healthy again: https://ci-beam.apache.org/job/beam_SeedJob/9762/ <https://ci-beam.apache.org/job/beam_SeedJob/9762/> .
>>
>> On Thu, Jun 2, 2022 at 10:24 AM Ryan Thompson <ryanthompson@google.com <ma...@google.com>> wrote:
>> I've filed a jira to infra and asked on the slack channel. Hopefully someone there has the power to restart the server.
>>
>> https://issues.apache.org/jira/browse/INFRA-23335 <https://issues.apache.org/jira/browse/INFRA-23335>
>>
>> On Thu, Jun 2, 2022 at 12:41 PM Kenneth Knowles <kenn@apache.org <ma...@apache.org>> wrote:
>> Commented on the Jira. Seems like it happens somewhat rarely, but is sometimes resolved by a restart, and sometimes has to do with some version mismatch issues. I did not find anything like a real root cause, just trial-and-error fixes. I'm not certain what may have occurred around May 19 to this infrastructure. Hoping INFRA can help us sort it out.
>>
>> Kenn
>>
>> On Thu, Jun 2, 2022 at 9:15 AM Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
>> Thanks Ahmet and Ryan for taking a look!
>>
>> I agree that referenced commits seems are not related, that us why I was puzzled with this.
>>
>>> On 2 Jun 2022, at 17:56, Ahmet Altay <altay@google.com <ma...@google.com>> wrote:
>>>
>>>
>>>
>>> On Thu, Jun 2, 2022 at 8:53 AM Ryan Thompson <ryanthompson@google.com <ma...@google.com>> wrote:
>>> I asked in the slack channel to restart jenkins. I'm looking through the past messages to see if there's someone there I can tag.
>>>
>>> Thank you. Unfortunately, I do not believe we have a single expert we can tag.
>>>
>>> Am I right in understanding this service is managed by someone in apache and not us?
>>>
>>> It is mixed. ASF Infra works with a vendor and they run a hosted jenkins instance. We control the configuration of the instance, and the worker nodes.
>>>
>>>
>>> On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <altay@google.com <ma...@google.com>> wrote:
>>> /cc @Kenneth Knowles <ma...@google.com>
>>> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <altay@google.com <ma...@google.com>> wrote:
>>> I do not have a great idea but googling about the error, similar errors were resolved by restarting jenkins. We could try that. We may need to ask infra. ( @Ryan Thompson <ma...@google.com> - could you please infra to restart jenkins?)
>>>
>>> I do not think this issue is related to any change in the source control. The commit referenced in the first failed job is an unrelated doc change.
>>>
>>>
>>> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
>>> I created a jira for this (not sure if it’s P0, but P1 for sure):
>>> https://issues.apache.org/jira/browse/BEAM-14548 <https://issues.apache.org/jira/browse/BEAM-14548>
>>>
>>> Could someone, who has more knowledge than me in Beam infrastructure/Jenkins, take a look, please?
>>>
>>> —
>>> Alexey
>>>
>>>> On 31 May 2022, at 17:08, Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
>>>>
>>>> The first failed job is https://ci-beam.apache.org/job/beam_SeedJob/9696/ <https://ci-beam.apache.org/job/beam_SeedJob/9696/>
>>>> It fails with this error (that says not so much):
>>>>
>>>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>>>> Processing DSL script .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>>>> Processing DSL script .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>>>> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
>>>> ERROR: java.io.IOException: Failed to persist config.xml
>>>>
>>>> and I don’t see any recent changes for these files.
>>>>
>>>>> On 31 May 2022, at 16:24, Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
>>>>>
>>>>> Hi everyone,
>>>>>
>>>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The last successful build [2] was 12 days ago.
>>>>> Does anyone know the reasons of this?
>>>>>
>>>>> —
>>>>> Alexey
>>>>>
>>>>>
>>>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/ <https://ci-beam.apache.org/job/beam_SeedJob/>
>>>>> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/ <https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Failing beam_SeedJob
Posted by Valentyn Tymofieiev <va...@google.com>.
I am not aware of a straightforward way. I have been using this script:
https://gist.github.com/tvalentyn/08cdb2836d311c55ae6ec09a276aea2a
+Pablo Estrada <pa...@google.com> investigated this in
https://issues.apache.org/jira/browse/BEAM-14377 but Pablo's changes were
rolled back because somehow seed job became broken.
On Fri, Jun 3, 2022 at 2:56 PM Alexey Romanenko <ar...@gmail.com>
wrote:
> Yes, it’s ok now.
> Thanks all!
>
> Side question - is there a way to run a Jenkins job against specific PR? I
> tried to specify different git refs as an input parameter but it didn’t
> work for me…
>
> —
> Alexey
>
> On 3 Jun 2022, at 05:02, Ahmet Altay <al...@google.com> wrote:
>
> Seed job looks healthy again:
> https://ci-beam.apache.org/job/beam_SeedJob/9762/ .
>
> On Thu, Jun 2, 2022 at 10:24 AM Ryan Thompson <ry...@google.com>
> wrote:
>
>> I've filed a jira to infra and asked on the slack channel. Hopefully
>> someone there has the power to restart the server.
>>
>> https://issues.apache.org/jira/browse/INFRA-23335
>>
>> On Thu, Jun 2, 2022 at 12:41 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> Commented on the Jira. Seems like it happens somewhat rarely, but is
>>> sometimes resolved by a restart, and sometimes has to do with some version
>>> mismatch issues. I did not find anything like a real root cause, just
>>> trial-and-error fixes. I'm not certain what may have occurred around May 19
>>> to this infrastructure. Hoping INFRA can help us sort it out.
>>>
>>> Kenn
>>>
>>> On Thu, Jun 2, 2022 at 9:15 AM Alexey Romanenko <
>>> aromanenko.dev@gmail.com> wrote:
>>>
>>>> Thanks Ahmet and Ryan for taking a look!
>>>>
>>>> I agree that referenced commits seems are not related, that us why I
>>>> was puzzled with this.
>>>>
>>>> On 2 Jun 2022, at 17:56, Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>
>>>>
>>>> On Thu, Jun 2, 2022 at 8:53 AM Ryan Thompson <ry...@google.com>
>>>> wrote:
>>>>
>>>>> I asked in the slack channel to restart jenkins. I'm looking through
>>>>> the past messages to see if there's someone there I can tag.
>>>>>
>>>>
>>>> Thank you. Unfortunately, I do not believe we have a single expert we
>>>> can tag.
>>>>
>>>>
>>>>> Am I right in understanding this service is managed by someone in
>>>>> apache and not us?
>>>>>
>>>>
>>>> It is mixed. ASF Infra works with a vendor and they run a hosted
>>>> jenkins instance. We control the configuration of the instance, and the
>>>> worker nodes.
>>>>
>>>>
>>>>>
>>>>> On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>> /cc @Kenneth Knowles <kl...@google.com>
>>>>>>
>>>>>> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>> I do not have a great idea but googling about the error, similar
>>>>>>> errors were resolved by restarting jenkins. We could try that. We may need
>>>>>>> to ask infra. ( @Ryan Thompson <ry...@google.com> - could
>>>>>>> you please infra to restart jenkins?)
>>>>>>>
>>>>>>> I do not think this issue is related to any change in the source
>>>>>>> control. The commit referenced in the first failed job is an unrelated doc
>>>>>>> change.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <
>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>
>>>>>>>> I created a jira for this (not sure if it’s P0, but P1 for sure):
>>>>>>>> https://issues.apache.org/jira/browse/BEAM-14548
>>>>>>>>
>>>>>>>> Could someone, who has more knowledge than me in Beam
>>>>>>>> infrastructure/Jenkins, take a look, please?
>>>>>>>>
>>>>>>>> —
>>>>>>>> Alexey
>>>>>>>>
>>>>>>>> On 31 May 2022, at 17:08, Alexey Romanenko <
>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>
>>>>>>>> The first failed job is
>>>>>>>> https://ci-beam.apache.org/job/beam_SeedJob/9696/
>>>>>>>> It fails with this error (that says not so much):
>>>>>>>>
>>>>>>>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>>>>>>>> Processing DSL script
>>>>>>>> .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>>>>>>>> Processing DSL script
>>>>>>>> .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>>>>>>>> Processing DSL script
>>>>>>>> .test-infra/jenkins/job_Dependency_Check.groovy
>>>>>>>> ERROR: java.io.IOException: Failed to persist config.xml
>>>>>>>>
>>>>>>>> and I don’t see any recent changes for these files.
>>>>>>>>
>>>>>>>> On 31 May 2022, at 16:24, Alexey Romanenko <
>>>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hi everyone,
>>>>>>>>
>>>>>>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th.
>>>>>>>> The last successful build [2] was 12 days ago.
>>>>>>>> Does anyone know the reasons of this?
>>>>>>>>
>>>>>>>> —
>>>>>>>> Alexey
>>>>>>>>
>>>>>>>>
>>>>>>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/
>>>>>>>> [2]
>>>>>>>> https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>
>
Re: Failing beam_SeedJob
Posted by Alexey Romanenko <ar...@gmail.com>.
Yes, it’s ok now.
Thanks all!
Side question - is there a way to run a Jenkins job against specific PR? I tried to specify different git refs as an input parameter but it didn’t work for me…
—
Alexey
> On 3 Jun 2022, at 05:02, Ahmet Altay <al...@google.com> wrote:
>
> Seed job looks healthy again: https://ci-beam.apache.org/job/beam_SeedJob/9762/ <https://ci-beam.apache.org/job/beam_SeedJob/9762/> .
>
> On Thu, Jun 2, 2022 at 10:24 AM Ryan Thompson <ryanthompson@google.com <ma...@google.com>> wrote:
> I've filed a jira to infra and asked on the slack channel. Hopefully someone there has the power to restart the server.
>
> https://issues.apache.org/jira/browse/INFRA-23335 <https://issues.apache.org/jira/browse/INFRA-23335>
>
> On Thu, Jun 2, 2022 at 12:41 PM Kenneth Knowles <kenn@apache.org <ma...@apache.org>> wrote:
> Commented on the Jira. Seems like it happens somewhat rarely, but is sometimes resolved by a restart, and sometimes has to do with some version mismatch issues. I did not find anything like a real root cause, just trial-and-error fixes. I'm not certain what may have occurred around May 19 to this infrastructure. Hoping INFRA can help us sort it out.
>
> Kenn
>
> On Thu, Jun 2, 2022 at 9:15 AM Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
> Thanks Ahmet and Ryan for taking a look!
>
> I agree that referenced commits seems are not related, that us why I was puzzled with this.
>
>> On 2 Jun 2022, at 17:56, Ahmet Altay <altay@google.com <ma...@google.com>> wrote:
>>
>>
>>
>> On Thu, Jun 2, 2022 at 8:53 AM Ryan Thompson <ryanthompson@google.com <ma...@google.com>> wrote:
>> I asked in the slack channel to restart jenkins. I'm looking through the past messages to see if there's someone there I can tag.
>>
>> Thank you. Unfortunately, I do not believe we have a single expert we can tag.
>>
>> Am I right in understanding this service is managed by someone in apache and not us?
>>
>> It is mixed. ASF Infra works with a vendor and they run a hosted jenkins instance. We control the configuration of the instance, and the worker nodes.
>>
>>
>> On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <altay@google.com <ma...@google.com>> wrote:
>> /cc @Kenneth Knowles <ma...@google.com>
>> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <altay@google.com <ma...@google.com>> wrote:
>> I do not have a great idea but googling about the error, similar errors were resolved by restarting jenkins. We could try that. We may need to ask infra. ( @Ryan Thompson <ma...@google.com> - could you please infra to restart jenkins?)
>>
>> I do not think this issue is related to any change in the source control. The commit referenced in the first failed job is an unrelated doc change.
>>
>>
>> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
>> I created a jira for this (not sure if it’s P0, but P1 for sure):
>> https://issues.apache.org/jira/browse/BEAM-14548 <https://issues.apache.org/jira/browse/BEAM-14548>
>>
>> Could someone, who has more knowledge than me in Beam infrastructure/Jenkins, take a look, please?
>>
>> —
>> Alexey
>>
>>> On 31 May 2022, at 17:08, Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
>>>
>>> The first failed job is https://ci-beam.apache.org/job/beam_SeedJob/9696/ <https://ci-beam.apache.org/job/beam_SeedJob/9696/>
>>> It fails with this error (that says not so much):
>>>
>>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>>> Processing DSL script .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>>> Processing DSL script .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>>> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
>>> ERROR: java.io.IOException: Failed to persist config.xml
>>>
>>> and I don’t see any recent changes for these files.
>>>
>>>> On 31 May 2022, at 16:24, Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
>>>>
>>>> Hi everyone,
>>>>
>>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The last successful build [2] was 12 days ago.
>>>> Does anyone know the reasons of this?
>>>>
>>>> —
>>>> Alexey
>>>>
>>>>
>>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/ <https://ci-beam.apache.org/job/beam_SeedJob/>
>>>> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/ <https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/>
>>>>
>>>>
>>>
>>
>
Re: Failing beam_SeedJob
Posted by Ahmet Altay <al...@google.com>.
Seed job looks healthy again:
https://ci-beam.apache.org/job/beam_SeedJob/9762/ .
On Thu, Jun 2, 2022 at 10:24 AM Ryan Thompson <ry...@google.com>
wrote:
> I've filed a jira to infra and asked on the slack channel. Hopefully
> someone there has the power to restart the server.
>
> https://issues.apache.org/jira/browse/INFRA-23335
>
> On Thu, Jun 2, 2022 at 12:41 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> Commented on the Jira. Seems like it happens somewhat rarely, but is
>> sometimes resolved by a restart, and sometimes has to do with some version
>> mismatch issues. I did not find anything like a real root cause, just
>> trial-and-error fixes. I'm not certain what may have occurred around May 19
>> to this infrastructure. Hoping INFRA can help us sort it out.
>>
>> Kenn
>>
>> On Thu, Jun 2, 2022 at 9:15 AM Alexey Romanenko <ar...@gmail.com>
>> wrote:
>>
>>> Thanks Ahmet and Ryan for taking a look!
>>>
>>> I agree that referenced commits seems are not related, that us why I was
>>> puzzled with this.
>>>
>>> On 2 Jun 2022, at 17:56, Ahmet Altay <al...@google.com> wrote:
>>>
>>>
>>>
>>> On Thu, Jun 2, 2022 at 8:53 AM Ryan Thompson <ry...@google.com>
>>> wrote:
>>>
>>>> I asked in the slack channel to restart jenkins. I'm looking through
>>>> the past messages to see if there's someone there I can tag.
>>>>
>>>
>>> Thank you. Unfortunately, I do not believe we have a single expert we
>>> can tag.
>>>
>>>
>>>> Am I right in understanding this service is managed by someone in
>>>> apache and not us?
>>>>
>>>
>>> It is mixed. ASF Infra works with a vendor and they run a hosted jenkins
>>> instance. We control the configuration of the instance, and the worker
>>> nodes.
>>>
>>>
>>>>
>>>> On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>> /cc @Kenneth Knowles <kl...@google.com>
>>>>>
>>>>> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>> I do not have a great idea but googling about the error, similar
>>>>>> errors were resolved by restarting jenkins. We could try that. We may need
>>>>>> to ask infra. ( @Ryan Thompson <ry...@google.com> - could you
>>>>>> please infra to restart jenkins?)
>>>>>>
>>>>>> I do not think this issue is related to any change in the source
>>>>>> control. The commit referenced in the first failed job is an unrelated doc
>>>>>> change.
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <
>>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>>
>>>>>>> I created a jira for this (not sure if it’s P0, but P1 for sure):
>>>>>>> https://issues.apache.org/jira/browse/BEAM-14548
>>>>>>>
>>>>>>> Could someone, who has more knowledge than me in Beam
>>>>>>> infrastructure/Jenkins, take a look, please?
>>>>>>>
>>>>>>> —
>>>>>>> Alexey
>>>>>>>
>>>>>>> On 31 May 2022, at 17:08, Alexey Romanenko <ar...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> The first failed job is
>>>>>>> https://ci-beam.apache.org/job/beam_SeedJob/9696/
>>>>>>> It fails with this error (that says not so much):
>>>>>>>
>>>>>>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>>>>>>> Processing DSL script
>>>>>>> .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>>>>>>> Processing DSL script
>>>>>>> .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>>>>>>> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
>>>>>>> ERROR: java.io.IOException: Failed to persist config.xml
>>>>>>>
>>>>>>> and I don’t see any recent changes for these files.
>>>>>>>
>>>>>>> On 31 May 2022, at 16:24, Alexey Romanenko <ar...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi everyone,
>>>>>>>
>>>>>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The
>>>>>>> last successful build [2] was 12 days ago.
>>>>>>> Does anyone know the reasons of this?
>>>>>>>
>>>>>>> —
>>>>>>> Alexey
>>>>>>>
>>>>>>>
>>>>>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/
>>>>>>> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>
Re: Failing beam_SeedJob
Posted by Ryan Thompson <ry...@google.com>.
I've filed a jira to infra and asked on the slack channel. Hopefully
someone there has the power to restart the server.
https://issues.apache.org/jira/browse/INFRA-23335
On Thu, Jun 2, 2022 at 12:41 PM Kenneth Knowles <ke...@apache.org> wrote:
> Commented on the Jira. Seems like it happens somewhat rarely, but is
> sometimes resolved by a restart, and sometimes has to do with some version
> mismatch issues. I did not find anything like a real root cause, just
> trial-and-error fixes. I'm not certain what may have occurred around May 19
> to this infrastructure. Hoping INFRA can help us sort it out.
>
> Kenn
>
> On Thu, Jun 2, 2022 at 9:15 AM Alexey Romanenko <ar...@gmail.com>
> wrote:
>
>> Thanks Ahmet and Ryan for taking a look!
>>
>> I agree that referenced commits seems are not related, that us why I was
>> puzzled with this.
>>
>> On 2 Jun 2022, at 17:56, Ahmet Altay <al...@google.com> wrote:
>>
>>
>>
>> On Thu, Jun 2, 2022 at 8:53 AM Ryan Thompson <ry...@google.com>
>> wrote:
>>
>>> I asked in the slack channel to restart jenkins. I'm looking through the
>>> past messages to see if there's someone there I can tag.
>>>
>>
>> Thank you. Unfortunately, I do not believe we have a single expert we can
>> tag.
>>
>>
>>> Am I right in understanding this service is managed by someone in apache
>>> and not us?
>>>
>>
>> It is mixed. ASF Infra works with a vendor and they run a hosted jenkins
>> instance. We control the configuration of the instance, and the worker
>> nodes.
>>
>>
>>>
>>> On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <al...@google.com> wrote:
>>>
>>>> /cc @Kenneth Knowles <kl...@google.com>
>>>>
>>>> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>> I do not have a great idea but googling about the error, similar
>>>>> errors were resolved by restarting jenkins. We could try that. We may need
>>>>> to ask infra. ( @Ryan Thompson <ry...@google.com> - could you
>>>>> please infra to restart jenkins?)
>>>>>
>>>>> I do not think this issue is related to any change in the source
>>>>> control. The commit referenced in the first failed job is an unrelated doc
>>>>> change.
>>>>>
>>>>>
>>>>> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <
>>>>> aromanenko.dev@gmail.com> wrote:
>>>>>
>>>>>> I created a jira for this (not sure if it’s P0, but P1 for sure):
>>>>>> https://issues.apache.org/jira/browse/BEAM-14548
>>>>>>
>>>>>> Could someone, who has more knowledge than me in Beam
>>>>>> infrastructure/Jenkins, take a look, please?
>>>>>>
>>>>>> —
>>>>>> Alexey
>>>>>>
>>>>>> On 31 May 2022, at 17:08, Alexey Romanenko <ar...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> The first failed job is
>>>>>> https://ci-beam.apache.org/job/beam_SeedJob/9696/
>>>>>> It fails with this error (that says not so much):
>>>>>>
>>>>>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>>>>>> Processing DSL script
>>>>>> .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>>>>>> Processing DSL script
>>>>>> .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>>>>>> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
>>>>>> ERROR: java.io.IOException: Failed to persist config.xml
>>>>>>
>>>>>> and I don’t see any recent changes for these files.
>>>>>>
>>>>>> On 31 May 2022, at 16:24, Alexey Romanenko <ar...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The
>>>>>> last successful build [2] was 12 days ago.
>>>>>> Does anyone know the reasons of this?
>>>>>>
>>>>>> —
>>>>>> Alexey
>>>>>>
>>>>>>
>>>>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/
>>>>>> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>
Re: Failing beam_SeedJob
Posted by Kenneth Knowles <ke...@apache.org>.
Commented on the Jira. Seems like it happens somewhat rarely, but is
sometimes resolved by a restart, and sometimes has to do with some version
mismatch issues. I did not find anything like a real root cause, just
trial-and-error fixes. I'm not certain what may have occurred around May 19
to this infrastructure. Hoping INFRA can help us sort it out.
Kenn
On Thu, Jun 2, 2022 at 9:15 AM Alexey Romanenko <ar...@gmail.com>
wrote:
> Thanks Ahmet and Ryan for taking a look!
>
> I agree that referenced commits seems are not related, that us why I was
> puzzled with this.
>
> On 2 Jun 2022, at 17:56, Ahmet Altay <al...@google.com> wrote:
>
>
>
> On Thu, Jun 2, 2022 at 8:53 AM Ryan Thompson <ry...@google.com>
> wrote:
>
>> I asked in the slack channel to restart jenkins. I'm looking through the
>> past messages to see if there's someone there I can tag.
>>
>
> Thank you. Unfortunately, I do not believe we have a single expert we can
> tag.
>
>
>> Am I right in understanding this service is managed by someone in apache
>> and not us?
>>
>
> It is mixed. ASF Infra works with a vendor and they run a hosted jenkins
> instance. We control the configuration of the instance, and the worker
> nodes.
>
>
>>
>> On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <al...@google.com> wrote:
>>
>>> /cc @Kenneth Knowles <kl...@google.com>
>>>
>>> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <al...@google.com> wrote:
>>>
>>>> I do not have a great idea but googling about the error, similar errors
>>>> were resolved by restarting jenkins. We could try that. We may need to ask
>>>> infra. ( @Ryan Thompson <ry...@google.com> - could you please
>>>> infra to restart jenkins?)
>>>>
>>>> I do not think this issue is related to any change in the source
>>>> control. The commit referenced in the first failed job is an unrelated doc
>>>> change.
>>>>
>>>>
>>>> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <
>>>> aromanenko.dev@gmail.com> wrote:
>>>>
>>>>> I created a jira for this (not sure if it’s P0, but P1 for sure):
>>>>> https://issues.apache.org/jira/browse/BEAM-14548
>>>>>
>>>>> Could someone, who has more knowledge than me in Beam
>>>>> infrastructure/Jenkins, take a look, please?
>>>>>
>>>>> —
>>>>> Alexey
>>>>>
>>>>> On 31 May 2022, at 17:08, Alexey Romanenko <ar...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> The first failed job is
>>>>> https://ci-beam.apache.org/job/beam_SeedJob/9696/
>>>>> It fails with this error (that says not so much):
>>>>>
>>>>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>>>>> Processing DSL script
>>>>> .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>>>>> Processing DSL script
>>>>> .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>>>>> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
>>>>> ERROR: java.io.IOException: Failed to persist config.xml
>>>>>
>>>>> and I don’t see any recent changes for these files.
>>>>>
>>>>> On 31 May 2022, at 16:24, Alexey Romanenko <ar...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi everyone,
>>>>>
>>>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The
>>>>> last successful build [2] was 12 days ago.
>>>>> Does anyone know the reasons of this?
>>>>>
>>>>> —
>>>>> Alexey
>>>>>
>>>>>
>>>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/
>>>>> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/
>>>>>
>>>>>
>>>>>
>>>>>
>
Re: Failing beam_SeedJob
Posted by Alexey Romanenko <ar...@gmail.com>.
Thanks Ahmet and Ryan for taking a look!
I agree that referenced commits seems are not related, that us why I was puzzled with this.
> On 2 Jun 2022, at 17:56, Ahmet Altay <al...@google.com> wrote:
>
>
>
> On Thu, Jun 2, 2022 at 8:53 AM Ryan Thompson <ryanthompson@google.com <ma...@google.com>> wrote:
> I asked in the slack channel to restart jenkins. I'm looking through the past messages to see if there's someone there I can tag.
>
> Thank you. Unfortunately, I do not believe we have a single expert we can tag.
>
> Am I right in understanding this service is managed by someone in apache and not us?
>
> It is mixed. ASF Infra works with a vendor and they run a hosted jenkins instance. We control the configuration of the instance, and the worker nodes.
>
>
> On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <altay@google.com <ma...@google.com>> wrote:
> /cc @Kenneth Knowles <ma...@google.com>
> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <altay@google.com <ma...@google.com>> wrote:
> I do not have a great idea but googling about the error, similar errors were resolved by restarting jenkins. We could try that. We may need to ask infra. ( @Ryan Thompson <ma...@google.com> - could you please infra to restart jenkins?)
>
> I do not think this issue is related to any change in the source control. The commit referenced in the first failed job is an unrelated doc change.
>
>
> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
> I created a jira for this (not sure if it’s P0, but P1 for sure):
> https://issues.apache.org/jira/browse/BEAM-14548 <https://issues.apache.org/jira/browse/BEAM-14548>
>
> Could someone, who has more knowledge than me in Beam infrastructure/Jenkins, take a look, please?
>
> —
> Alexey
>
>> On 31 May 2022, at 17:08, Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
>>
>> The first failed job is https://ci-beam.apache.org/job/beam_SeedJob/9696/ <https://ci-beam.apache.org/job/beam_SeedJob/9696/>
>> It fails with this error (that says not so much):
>>
>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>> Processing DSL script .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>> Processing DSL script .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
>> ERROR: java.io.IOException: Failed to persist config.xml
>>
>> and I don’t see any recent changes for these files.
>>
>>> On 31 May 2022, at 16:24, Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
>>>
>>> Hi everyone,
>>>
>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The last successful build [2] was 12 days ago.
>>> Does anyone know the reasons of this?
>>>
>>> —
>>> Alexey
>>>
>>>
>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/ <https://ci-beam.apache.org/job/beam_SeedJob/>
>>> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/ <https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/>
>>>
>>>
>>
>
Re: Failing beam_SeedJob
Posted by Ahmet Altay <al...@google.com>.
On Thu, Jun 2, 2022 at 8:53 AM Ryan Thompson <ry...@google.com>
wrote:
> I asked in the slack channel to restart jenkins. I'm looking through the
> past messages to see if there's someone there I can tag.
>
Thank you. Unfortunately, I do not believe we have a single expert we can
tag.
> Am I right in understanding this service is managed by someone in apache
> and not us?
>
It is mixed. ASF Infra works with a vendor and they run a hosted jenkins
instance. We control the configuration of the instance, and the worker
nodes.
>
> On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <al...@google.com> wrote:
>
>> /cc @Kenneth Knowles <kl...@google.com>
>>
>> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <al...@google.com> wrote:
>>
>>> I do not have a great idea but googling about the error, similar errors
>>> were resolved by restarting jenkins. We could try that. We may need to ask
>>> infra. ( @Ryan Thompson <ry...@google.com> - could you please
>>> infra to restart jenkins?)
>>>
>>> I do not think this issue is related to any change in the source
>>> control. The commit referenced in the first failed job is an unrelated doc
>>> change.
>>>
>>>
>>> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <
>>> aromanenko.dev@gmail.com> wrote:
>>>
>>>> I created a jira for this (not sure if it’s P0, but P1 for sure):
>>>> https://issues.apache.org/jira/browse/BEAM-14548
>>>>
>>>> Could someone, who has more knowledge than me in Beam
>>>> infrastructure/Jenkins, take a look, please?
>>>>
>>>> —
>>>> Alexey
>>>>
>>>> On 31 May 2022, at 17:08, Alexey Romanenko <ar...@gmail.com>
>>>> wrote:
>>>>
>>>> The first failed job is
>>>> https://ci-beam.apache.org/job/beam_SeedJob/9696/
>>>> It fails with this error (that says not so much):
>>>>
>>>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>>>> Processing DSL script
>>>> .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>>>> Processing DSL script
>>>> .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>>>> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
>>>> ERROR: java.io.IOException: Failed to persist config.xml
>>>>
>>>> and I don’t see any recent changes for these files.
>>>>
>>>> On 31 May 2022, at 16:24, Alexey Romanenko <ar...@gmail.com>
>>>> wrote:
>>>>
>>>> Hi everyone,
>>>>
>>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The
>>>> last successful build [2] was 12 days ago.
>>>> Does anyone know the reasons of this?
>>>>
>>>> —
>>>> Alexey
>>>>
>>>>
>>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/
>>>> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/
>>>>
>>>>
>>>>
>>>>
Re: Failing beam_SeedJob
Posted by Ryan Thompson <ry...@google.com>.
I asked in the slack channel to restart jenkins. I'm looking through the
past messages to see if there's someone there I can tag. Am I right in
understanding this service is managed by someone in apache and not us?
On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <al...@google.com> wrote:
> /cc @Kenneth Knowles <kl...@google.com>
>
> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <al...@google.com> wrote:
>
>> I do not have a great idea but googling about the error, similar errors
>> were resolved by restarting jenkins. We could try that. We may need to ask
>> infra. ( @Ryan Thompson <ry...@google.com> - could you please
>> infra to restart jenkins?)
>>
>> I do not think this issue is related to any change in the source control.
>> The commit referenced in the first failed job is an unrelated doc change.
>>
>>
>> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <ar...@gmail.com>
>> wrote:
>>
>>> I created a jira for this (not sure if it’s P0, but P1 for sure):
>>> https://issues.apache.org/jira/browse/BEAM-14548
>>>
>>> Could someone, who has more knowledge than me in Beam
>>> infrastructure/Jenkins, take a look, please?
>>>
>>> —
>>> Alexey
>>>
>>> On 31 May 2022, at 17:08, Alexey Romanenko <ar...@gmail.com>
>>> wrote:
>>>
>>> The first failed job is
>>> https://ci-beam.apache.org/job/beam_SeedJob/9696/
>>> It fails with this error (that says not so much):
>>>
>>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>>> Processing DSL script
>>> .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>>> Processing DSL script
>>> .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>>> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
>>> ERROR: java.io.IOException: Failed to persist config.xml
>>>
>>> and I don’t see any recent changes for these files.
>>>
>>> On 31 May 2022, at 16:24, Alexey Romanenko <ar...@gmail.com>
>>> wrote:
>>>
>>> Hi everyone,
>>>
>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The
>>> last successful build [2] was 12 days ago.
>>> Does anyone know the reasons of this?
>>>
>>> —
>>> Alexey
>>>
>>>
>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/
>>> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/
>>>
>>>
>>>
>>>
Re: Failing beam_SeedJob
Posted by Ahmet Altay <al...@google.com>.
/cc @Kenneth Knowles <kl...@google.com>
On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <al...@google.com> wrote:
> I do not have a great idea but googling about the error, similar errors
> were resolved by restarting jenkins. We could try that. We may need to ask
> infra. ( @Ryan Thompson <ry...@google.com> - could you please
> infra to restart jenkins?)
>
> I do not think this issue is related to any change in the source control.
> The commit referenced in the first failed job is an unrelated doc change.
>
>
> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <ar...@gmail.com>
> wrote:
>
>> I created a jira for this (not sure if it’s P0, but P1 for sure):
>> https://issues.apache.org/jira/browse/BEAM-14548
>>
>> Could someone, who has more knowledge than me in Beam
>> infrastructure/Jenkins, take a look, please?
>>
>> —
>> Alexey
>>
>> On 31 May 2022, at 17:08, Alexey Romanenko <ar...@gmail.com>
>> wrote:
>>
>> The first failed job is https://ci-beam.apache.org/job/beam_SeedJob/9696/
>> It fails with this error (that says not so much):
>>
>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
>> Processing DSL script
>> .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>> Processing DSL script
>> .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
>> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
>> ERROR: java.io.IOException: Failed to persist config.xml
>>
>> and I don’t see any recent changes for these files.
>>
>> On 31 May 2022, at 16:24, Alexey Romanenko <ar...@gmail.com>
>> wrote:
>>
>> Hi everyone,
>>
>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The last
>> successful build [2] was 12 days ago.
>> Does anyone know the reasons of this?
>>
>> —
>> Alexey
>>
>>
>> [1] https://ci-beam.apache.org/job/beam_SeedJob/
>> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/
>>
>>
>>
>>
Re: Failing beam_SeedJob
Posted by Ahmet Altay <al...@google.com>.
I do not have a great idea but googling about the error, similar errors
were resolved by restarting jenkins. We could try that. We may need to ask
infra. ( @Ryan Thompson <ry...@google.com> - could you please infra
to restart jenkins?)
I do not think this issue is related to any change in the source control.
The commit referenced in the first failed job is an unrelated doc change.
On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko <ar...@gmail.com>
wrote:
> I created a jira for this (not sure if it’s P0, but P1 for sure):
> https://issues.apache.org/jira/browse/BEAM-14548
>
> Could someone, who has more knowledge than me in Beam
> infrastructure/Jenkins, take a look, please?
>
> —
> Alexey
>
> On 31 May 2022, at 17:08, Alexey Romanenko <ar...@gmail.com>
> wrote:
>
> The first failed job is https://ci-beam.apache.org/job/beam_SeedJob/9696/
> It fails with this error (that says not so much):
>
> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
> Processing DSL script
> .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
> Processing DSL script
> .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
> ERROR: java.io.IOException: Failed to persist config.xml
>
> and I don’t see any recent changes for these files.
>
> On 31 May 2022, at 16:24, Alexey Romanenko <ar...@gmail.com>
> wrote:
>
> Hi everyone,
>
> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The last
> successful build [2] was 12 days ago.
> Does anyone know the reasons of this?
>
> —
> Alexey
>
>
> [1] https://ci-beam.apache.org/job/beam_SeedJob/
> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/
>
>
>
>
Re: Failing beam_SeedJob
Posted by Alexey Romanenko <ar...@gmail.com>.
I created a jira for this (not sure if it’s P0, but P1 for sure):
https://issues.apache.org/jira/browse/BEAM-14548 <https://issues.apache.org/jira/browse/BEAM-14548>
Could someone, who has more knowledge than me in Beam infrastructure/Jenkins, take a look, please?
—
Alexey
> On 31 May 2022, at 17:08, Alexey Romanenko <ar...@gmail.com> wrote:
>
> The first failed job is https://ci-beam.apache.org/job/beam_SeedJob/9696/ <https://ci-beam.apache.org/job/beam_SeedJob/9696/>
> It fails with this error (that says not so much):
>
> Processing DSL script .test-infra/jenkins/job_00_seed.groovy
> Processing DSL script .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
> Processing DSL script .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
> ERROR: java.io.IOException: Failed to persist config.xml
>
> and I don’t see any recent changes for these files.
>
>> On 31 May 2022, at 16:24, Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hi everyone,
>>
>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The last successful build [2] was 12 days ago.
>> Does anyone know the reasons of this?
>>
>> —
>> Alexey
>>
>>
>> [1] https://ci-beam.apache.org/job/beam_SeedJob/ <https://ci-beam.apache.org/job/beam_SeedJob/>
>> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/ <https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/>
>>
>>
>
Re: Failing beam_SeedJob
Posted by Alexey Romanenko <ar...@gmail.com>.
The first failed job is https://ci-beam.apache.org/job/beam_SeedJob/9696/ <https://ci-beam.apache.org/job/beam_SeedJob/9696/>
It fails with this error (that says not so much):
Processing DSL script .test-infra/jenkins/job_00_seed.groovy
Processing DSL script .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
Processing DSL script .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
ERROR: java.io.IOException: Failed to persist config.xml
and I don’t see any recent changes for these files.
> On 31 May 2022, at 16:24, Alexey Romanenko <ar...@gmail.com> wrote:
>
> Hi everyone,
>
> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The last successful build [2] was 12 days ago.
> Does anyone know the reasons of this?
>
> —
> Alexey
>
>
> [1] https://ci-beam.apache.org/job/beam_SeedJob/ <https://ci-beam.apache.org/job/beam_SeedJob/>
> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/ <https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/>
>
>