You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Julian Hyde <jh...@apache.org> on 2021/04/14 17:39:20 UTC
Reset/restart CI
The same CI job seems to be failing in very similar ways [1]. I
suspect there is some "memory" in the CI environment causing this
error:
> java.net.BindException: Cannot assign requested address: Service 'sparkDriver' failed
> after 16 retries (on a random free port)! Consider explicitly setting the appropriate
> binding address for the service 'sparkDriver' (for example spark.driver.bindAddress
> for SparkDriver) to the correct binding address.
How to reset the CI environment?
Julian
[1] https://github.com/julianhyde/calcite/runs/2338351683?check_suite_focus=true
Re: Reset/restart CI
Posted by Julian Hyde <jh...@gmail.com>.
I see that Stamatis merged the PR. Thanks for the PR!
> On Apr 18, 2021, at 7:44 PM, JiaTao Tao <ta...@gmail.com> wrote:
>
> PR is already created and UT&CI are all passed for a few days:
> https://github.com/apache/calcite/pull/2404.
> About the Github ENV, I think we are not authorized to do this.
> The PR merge is blocked for some days, let's done this ASAP.
>
>
> Regards!
>
> Aron Tao
>
>
> Julian Hyde <jh...@gmail.com> 于2021年4月19日周一 上午3:21写道：
>
>> Aron & Alessandro,
>>
>> Can you discuss & come to consensus in
>> https://issues.apache.org/jira/browse/CALCITE-4587 <
>> https://issues.apache.org/jira/browse/CALCITE-4587>, and create a PR.
>>
>> Julian
>>
>>
>>
>>> On Apr 17, 2021, at 10:12 PM, JiaTao Tao <ta...@gmail.com> wrote:
>>>
>>> Yes, I believe set ENV can archive the same purpose, the benefit of
>> setting
>>> this in code is more stable and unaffected by the environment, as
>> mentioned
>>> above: "GitHub updates virtual environments from time to time".
>>>
>>>
>>> Regards!
>>>
>>> Aron Tao
>>>
>>>
>>> Alessandro Solimando <al...@gmail.com> 于2021年4月16日周五
>>> 下午8:55写道：
>>>
>>>> Hi all,
>>>> locally on MacOS I have to set the SPARK_LOCAL_IP environment variable
>> to
>>>> overcome that very same problem (something along the line of export
>>>> SPARK_LOCAL_IP="127.0.0.1").
>>>>
>>>> I wonder if it wouldn't be less intrusive to set an environment variable
>>>> rather than statically setting a SparkConf option in the code as Aron is
>>>> suggesting.
>>>>
>>>> Best regards,
>>>> Alessandro
>>>>
>>>> On Fri, 16 Apr 2021 at 12:26, JiaTao Tao <ta...@gmail.com> wrote:
>>>>
>>>>> Hi
>>>>> The problem may be the wrong hostname in "/etc/hosts", a way to solve
>>>> this
>>>>> problem is to set "spark.driver.bindAddress" explicitly, I've tested
>> this
>>>>> and it works.
>>>>> I've created a JIRA(CALCITE-4587) and attached the PR(
>>>>> https://github.com/apache/calcite/pull/2404/files).
>>>>>
>>>>> Regards!
>>>>>
>>>>> Aron Tao
>>>>>
>>>>>
>>>>> Vladimir Sitnikov <si...@gmail.com> 于2021年4月15日周四
>> 上午2:48写道：
>>>>>
>>>>>>> java.net.BindException: Cannot assign requested address: Service
>>>>>> 'sparkDriver' failed
>>>>>>> after 16 retries (on a random free port)! Consider explicitly
>>>> setting
>>>>>> the appropriate
>>>>>>> binding address for the service 'sparkDriver' (for example
>>>>>> spark.driver.bindAddress
>>>>>>> for SparkDriver) to the correct binding address.
>>>>>>
>>>>>> GitHub updates virtual environments from time to time, so the failure
>>>>> might
>>>>>> be
>>>>>> related to a new environment having a different set of network
>>>>> interfaces.
>>>>>>
>>>>>> The failure looks like a true bug rather than a CI glitch.
>>>>>>
>>>>>> Vladimir
>>>>>>
>>>>>
>>>>
>>
>>
Re: Reset/restart CI
Posted by JiaTao Tao <ta...@gmail.com>.
PR is already created and UT&CI are all passed for a few days:
https://github.com/apache/calcite/pull/2404.
About the Github ENV, I think we are not authorized to do this.
The PR merge is blocked for some days, let's done this ASAP.
Regards!
Aron Tao
Julian Hyde <jh...@gmail.com> 于2021年4月19日周一 上午3:21写道：
> Aron & Alessandro,
>
> Can you discuss & come to consensus in
> https://issues.apache.org/jira/browse/CALCITE-4587 <
> https://issues.apache.org/jira/browse/CALCITE-4587>, and create a PR.
>
> Julian
>
>
>
> > On Apr 17, 2021, at 10:12 PM, JiaTao Tao <ta...@gmail.com> wrote:
> >
> > Yes, I believe set ENV can archive the same purpose, the benefit of
> setting
> > this in code is more stable and unaffected by the environment, as
> mentioned
> > above: "GitHub updates virtual environments from time to time".
> >
> >
> > Regards!
> >
> > Aron Tao
> >
> >
> > Alessandro Solimando <al...@gmail.com> 于2021年4月16日周五
> > 下午8:55写道：
> >
> >> Hi all,
> >> locally on MacOS I have to set the SPARK_LOCAL_IP environment variable
> to
> >> overcome that very same problem (something along the line of export
> >> SPARK_LOCAL_IP="127.0.0.1").
> >>
> >> I wonder if it wouldn't be less intrusive to set an environment variable
> >> rather than statically setting a SparkConf option in the code as Aron is
> >> suggesting.
> >>
> >> Best regards,
> >> Alessandro
> >>
> >> On Fri, 16 Apr 2021 at 12:26, JiaTao Tao <ta...@gmail.com> wrote:
> >>
> >>> Hi
> >>> The problem may be the wrong hostname in "/etc/hosts", a way to solve
> >> this
> >>> problem is to set "spark.driver.bindAddress" explicitly, I've tested
> this
> >>> and it works.
> >>> I've created a JIRA(CALCITE-4587) and attached the PR(
> >>> https://github.com/apache/calcite/pull/2404/files).
> >>>
> >>> Regards!
> >>>
> >>> Aron Tao
> >>>
> >>>
> >>> Vladimir Sitnikov <si...@gmail.com> 于2021年4月15日周四
> 上午2:48写道：
> >>>
> >>>>> java.net.BindException: Cannot assign requested address: Service
> >>>> 'sparkDriver' failed
> >>>>> after 16 retries (on a random free port)! Consider explicitly
> >> setting
> >>>> the appropriate
> >>>>> binding address for the service 'sparkDriver' (for example
> >>>> spark.driver.bindAddress
> >>>>> for SparkDriver) to the correct binding address.
> >>>>
> >>>> GitHub updates virtual environments from time to time, so the failure
> >>> might
> >>>> be
> >>>> related to a new environment having a different set of network
> >>> interfaces.
> >>>>
> >>>> The failure looks like a true bug rather than a CI glitch.
> >>>>
> >>>> Vladimir
> >>>>
> >>>
> >>
>
>
Re: Reset/restart CI
Posted by Julian Hyde <jh...@gmail.com>.
Aron & Alessandro,
Can you discuss & come to consensus in https://issues.apache.org/jira/browse/CALCITE-4587 <https://issues.apache.org/jira/browse/CALCITE-4587>, and create a PR.
Julian
> On Apr 17, 2021, at 10:12 PM, JiaTao Tao <ta...@gmail.com> wrote:
>
> Yes, I believe set ENV can archive the same purpose, the benefit of setting
> this in code is more stable and unaffected by the environment, as mentioned
> above: "GitHub updates virtual environments from time to time".
>
>
> Regards!
>
> Aron Tao
>
>
> Alessandro Solimando <al...@gmail.com> 于2021年4月16日周五
> 下午8:55写道：
>
>> Hi all,
>> locally on MacOS I have to set the SPARK_LOCAL_IP environment variable to
>> overcome that very same problem (something along the line of export
>> SPARK_LOCAL_IP="127.0.0.1").
>>
>> I wonder if it wouldn't be less intrusive to set an environment variable
>> rather than statically setting a SparkConf option in the code as Aron is
>> suggesting.
>>
>> Best regards,
>> Alessandro
>>
>> On Fri, 16 Apr 2021 at 12:26, JiaTao Tao <ta...@gmail.com> wrote:
>>
>>> Hi
>>> The problem may be the wrong hostname in "/etc/hosts", a way to solve
>> this
>>> problem is to set "spark.driver.bindAddress" explicitly, I've tested this
>>> and it works.
>>> I've created a JIRA(CALCITE-4587) and attached the PR(
>>> https://github.com/apache/calcite/pull/2404/files).
>>>
>>> Regards!
>>>
>>> Aron Tao
>>>
>>>
>>> Vladimir Sitnikov <si...@gmail.com> 于2021年4月15日周四 上午2:48写道：
>>>
>>>>> java.net.BindException: Cannot assign requested address: Service
>>>> 'sparkDriver' failed
>>>>> after 16 retries (on a random free port)! Consider explicitly
>> setting
>>>> the appropriate
>>>>> binding address for the service 'sparkDriver' (for example
>>>> spark.driver.bindAddress
>>>>> for SparkDriver) to the correct binding address.
>>>>
>>>> GitHub updates virtual environments from time to time, so the failure
>>> might
>>>> be
>>>> related to a new environment having a different set of network
>>> interfaces.
>>>>
>>>> The failure looks like a true bug rather than a CI glitch.
>>>>
>>>> Vladimir
>>>>
>>>
>>
Re: Reset/restart CI
Posted by JiaTao Tao <ta...@gmail.com>.
Yes, I believe set ENV can archive the same purpose, the benefit of setting
this in code is more stable and unaffected by the environment, as mentioned
above: "GitHub updates virtual environments from time to time".
Regards!
Aron Tao
Alessandro Solimando <al...@gmail.com> 于2021年4月16日周五
下午8:55写道：
> Hi all,
> locally on MacOS I have to set the SPARK_LOCAL_IP environment variable to
> overcome that very same problem (something along the line of export
> SPARK_LOCAL_IP="127.0.0.1").
>
> I wonder if it wouldn't be less intrusive to set an environment variable
> rather than statically setting a SparkConf option in the code as Aron is
> suggesting.
>
> Best regards,
> Alessandro
>
> On Fri, 16 Apr 2021 at 12:26, JiaTao Tao <ta...@gmail.com> wrote:
>
> > Hi
> > The problem may be the wrong hostname in "/etc/hosts", a way to solve
> this
> > problem is to set "spark.driver.bindAddress" explicitly, I've tested this
> > and it works.
> > I've created a JIRA(CALCITE-4587) and attached the PR(
> > https://github.com/apache/calcite/pull/2404/files).
> >
> > Regards!
> >
> > Aron Tao
> >
> >
> > Vladimir Sitnikov <si...@gmail.com> 于2021年4月15日周四 上午2:48写道：
> >
> > > > java.net.BindException: Cannot assign requested address: Service
> > > 'sparkDriver' failed
> > > > after 16 retries (on a random free port)! Consider explicitly
> setting
> > > the appropriate
> > > > binding address for the service 'sparkDriver' (for example
> > > spark.driver.bindAddress
> > > > for SparkDriver) to the correct binding address.
> > >
> > > GitHub updates virtual environments from time to time, so the failure
> > might
> > > be
> > > related to a new environment having a different set of network
> > interfaces.
> > >
> > > The failure looks like a true bug rather than a CI glitch.
> > >
> > > Vladimir
> > >
> >
>
Re: Reset/restart CI
Posted by Alessandro Solimando <al...@gmail.com>.
Hi all,
locally on MacOS I have to set the SPARK_LOCAL_IP environment variable to
overcome that very same problem (something along the line of export
SPARK_LOCAL_IP="127.0.0.1").
I wonder if it wouldn't be less intrusive to set an environment variable
rather than statically setting a SparkConf option in the code as Aron is
suggesting.
Best regards,
Alessandro
On Fri, 16 Apr 2021 at 12:26, JiaTao Tao <ta...@gmail.com> wrote:
> Hi
> The problem may be the wrong hostname in "/etc/hosts", a way to solve this
> problem is to set "spark.driver.bindAddress" explicitly, I've tested this
> and it works.
> I've created a JIRA(CALCITE-4587) and attached the PR(
> https://github.com/apache/calcite/pull/2404/files).
>
> Regards!
>
> Aron Tao
>
>
> Vladimir Sitnikov <si...@gmail.com> 于2021年4月15日周四 上午2:48写道：
>
> > > java.net.BindException: Cannot assign requested address: Service
> > 'sparkDriver' failed
> > > after 16 retries (on a random free port)! Consider explicitly setting
> > the appropriate
> > > binding address for the service 'sparkDriver' (for example
> > spark.driver.bindAddress
> > > for SparkDriver) to the correct binding address.
> >
> > GitHub updates virtual environments from time to time, so the failure
> might
> > be
> > related to a new environment having a different set of network
> interfaces.
> >
> > The failure looks like a true bug rather than a CI glitch.
> >
> > Vladimir
> >
>
Re: Reset/restart CI
Posted by JiaTao Tao <ta...@gmail.com>.
Hi
The problem may be the wrong hostname in "/etc/hosts", a way to solve this
problem is to set "spark.driver.bindAddress" explicitly, I've tested this
and it works.
I've created a JIRA(CALCITE-4587) and attached the PR(
https://github.com/apache/calcite/pull/2404/files).
Regards!
Aron Tao
Vladimir Sitnikov <si...@gmail.com> 于2021年4月15日周四 上午2:48写道：
> > java.net.BindException: Cannot assign requested address: Service
> 'sparkDriver' failed
> > after 16 retries (on a random free port)! Consider explicitly setting
> the appropriate
> > binding address for the service 'sparkDriver' (for example
> spark.driver.bindAddress
> > for SparkDriver) to the correct binding address.
>
> GitHub updates virtual environments from time to time, so the failure might
> be
> related to a new environment having a different set of network interfaces.
>
> The failure looks like a true bug rather than a CI glitch.
>
> Vladimir
>
Re: Reset/restart CI
Posted by Vladimir Sitnikov <si...@gmail.com>.
> java.net.BindException: Cannot assign requested address: Service
'sparkDriver' failed
> after 16 retries (on a random free port)! Consider explicitly setting
the appropriate
> binding address for the service 'sparkDriver' (for example
spark.driver.bindAddress
> for SparkDriver) to the correct binding address.
GitHub updates virtual environments from time to time, so the failure might
be
related to a new environment having a different set of network interfaces.
The failure looks like a true bug rather than a CI glitch.
Vladimir