You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Shane Knapp <sk...@berkeley.edu> on 2019/10/07 18:31:56 UTC
[build system] maven master branch builds timing out en masse...
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7/
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7-ubuntu-testing/
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2/
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2-jdk-11/
spark-master-test-maven-hadoop-3.2 looks like it's been timing out for
~2 weeks, but the others only for a few days.
all of these builds have a 335 minute (5 hours, 35 mins) timeout configured.
honestly, i feel that 5.5 hours is already an insane amount of time
for these builds to run... do we need to increase it? can
someone(s) here figure out what's taking so long and refactor some of
the tests?
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [build system] maven master branch builds timing out en masse...
Posted by Sean Owen <sr...@gmail.com>.
Moving the conversation here -- yes, why on earth are they taking this
long all of the sudden? we'll have to look again when they come back
online. The last successful build took 6 hours, of which 4:45 were
the unit tests themselves.
It's mostly SQL tests; SQLQuerySuite is approaching an hour.
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2/lastStableBuild/testReport/org.apache.spark.sql/SQLQueryTestSuite/
Well, at least it seems like we legitimately need these full
integration tests to run that long, and that's just life right now, so
8 hours is better than failing. But yeah, seems like we just test a
whole lot more recently and it's taking a long time. Kudos to recent
efforts to parallelize some of that.
On Mon, Oct 7, 2019 at 1:52 PM Shane Knapp <sk...@berkeley.edu> wrote:
>
> just chatted w/sean privately and i'm going to up the test timeouts to
> 480mins (8 hours).
>
> i still don't like this but at least it should hopefully get things green again.
>
> On Mon, Oct 7, 2019 at 11:31 AM Shane Knapp <sk...@berkeley.edu> wrote:
> >
> > https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7/
> > https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7-ubuntu-testing/
> > https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2/
> > https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2-jdk-11/
> >
> > spark-master-test-maven-hadoop-3.2 looks like it's been timing out for
> > ~2 weeks, but the others only for a few days.
> >
> > all of these builds have a 335 minute (5 hours, 35 mins) timeout configured.
> >
> > honestly, i feel that 5.5 hours is already an insane amount of time
> > for these builds to run... do we need to increase it? can
> > someone(s) here figure out what's taking so long and refactor some of
> > the tests?
> >
> > shane
> > --
> > Shane Knapp
> > UC Berkeley EECS Research / RISELab Staff Technical Lead
> > https://rise.cs.berkeley.edu
>
>
>
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [build system] maven master branch builds timing out en masse...
Posted by Shane Knapp <sk...@berkeley.edu>.
just chatted w/sean privately and i'm going to up the test timeouts to
480mins (8 hours).
i still don't like this but at least it should hopefully get things green again.
On Mon, Oct 7, 2019 at 11:31 AM Shane Knapp <sk...@berkeley.edu> wrote:
>
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7/
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7-ubuntu-testing/
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2/
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2-jdk-11/
>
> spark-master-test-maven-hadoop-3.2 looks like it's been timing out for
> ~2 weeks, but the others only for a few days.
>
> all of these builds have a 335 minute (5 hours, 35 mins) timeout configured.
>
> honestly, i feel that 5.5 hours is already an insane amount of time
> for these builds to run... do we need to increase it? can
> someone(s) here figure out what's taking so long and refactor some of
> the tests?
>
> shane
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org