You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Kenneth Knowles <kl...@google.com.INVALID> on 2016/12/13 01:36:17 UTC
Jenkins pre/postcommit increased from 35m to 60m+ on Friday
Hi all,
We have a huge Jenkins backlog, surely exacerbated by the fact that our
test time (precommit and postcommit mvn install) has roughly doubled in the
last few days.
Here's the quick link to the trend:
https://builds.apache.org/view/Beam/job/beam_PostCommit_Java_MavenInstall/buildTimeTrend
Good 33m build at 2016-12-09 02:42:
https://builds.apache.org/view/Beam/job/beam_PostCommit_Java_MavenInstall/2041/
Bad 59m build at 2016-12-09 18:00 (trigger by timer):
https://builds.apache.org/view/Beam/job/beam_PostCommit_Java_MavenInstall/2048/
There are a couple middling runs in between that I can't place immediately
into either bucket. I'm still looking into this, but it is a clicky-clicky
process that hasn't yielded anything yet.
If anyone knows something, please share your insight.
Kenn
Re: Jenkins pre/postcommit increased from 35m to 60m+ on Friday
Posted by Dan Halperin <dh...@google.com.INVALID>.
status.apache.org is an excellent resource:
http://status.apache.org/#wplaceholder_86
It does indeed look like Jenkins has been having issues for nearly a week.
On Mon, Dec 12, 2016 at 6:32 PM, Kenneth Knowles <kl...@google.com.invalid>
wrote:
> Great. That means the timestamp change I made for Travis, ported to
> Jenkins, should reveal more.
>
> Meanwhile - any known issues with Jenkins or Maven Central? Status
> dashboard for Maven Central doesn't look unhappy.
>
> On Mon, Dec 12, 2016 at 6:25 PM, Dan Halperin <dhalperi@google.com.invalid
> >
> wrote:
>
> > From the "bad run", the Maven part took 35 minutes and presumably the
> rest
> > is Jenkins / Maven / downloading overhead.
> >
> > [INFO] ------------------------------------------------------------
> > ------------
> > [INFO] Reactor Summary:
> > [INFO]
> > [INFO] Apache Beam :: Parent .............................. SUCCESS [
> > 37.558 s]
> > [INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [
> > 8.172 s]
> > [INFO] Apache Beam :: SDKs ................................ SUCCESS [
> > 11.521 s]
> > [INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [
> > 10.801 s]
> > [INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS
> [04:04
> > min]
> > [INFO] Apache Beam :: Runners ............................. SUCCESS [
> > 14.945 s]
> > [INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [
> > 44.654 s]
> > [INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS
> [02:06
> > min]
> > [INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [
> > 33.245 s]
> > [INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [
> > 4.047 s]
> > [INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform
> > SUCCESS [04:32 min]
> > [INFO] Apache Beam :: SDKs :: Java :: IO :: HDFS .......... SUCCESS [
> > 32.009 s]
> > [INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [
> > 19.006 s]
> > [INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [
> > 22.021 s]
> > [INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [
> > 22.817 s]
> > [INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [
> > 27.276 s]
> > [INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [
> > 23.662 s]
> > [INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [
> > 3.115 s]
> > [INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter
> > SUCCESS [ 17.818 s]
> > [INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples
> > SUCCESS [ 15.111 s]
> > [INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples -
> > Java 8 SUCCESS [ 24.477 s]
> > [INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [
> > 5.759 s]
> > [INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library
> > SUCCESS [ 22.293 s]
> > [INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [
> > 31.145 s]
> > [INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [
> > 8.033 s]
> > [INFO] Apache Beam :: Runners :: Flink .................... SUCCESS [
> > 6.503 s]
> > [INFO] Apache Beam :: Runners :: Flink :: Core ............ SUCCESS [
> > 43.593 s]
> > [INFO] Apache Beam :: Runners :: Flink :: Examples ........ SUCCESS [
> > 20.006 s]
> > [INFO] Apache Beam :: Runners :: Spark .................... SUCCESS
> [03:10
> > min]
> > [INFO] Apache Beam :: Runners :: Apex ..................... SUCCESS
> [01:03
> > min]
> > [INFO] Apache Beam :: Examples ............................ SUCCESS [
> > 8.124 s]
> > [INFO] Apache Beam :: Examples :: Java .................... SUCCESS
> [05:29
> > min]
> > [INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [
> > 19.005 s]
> > [INFO] ------------------------------------------------------------
> > ------------
> > [INFO] BUILD SUCCESS
> > [INFO] ------------------------------------------------------------
> > ------------
> > [INFO] Total time: 34:30 min
> > [INFO] Finished at: 2016-12-09T18:50:49+00:00
> > [INFO] Final Memory: 196M/1051M
> > [INFO] ------------------------------------------------------------
> > ------------
> >
> >
> >
> > On Mon, Dec 12, 2016 at 5:36 PM, Kenneth Knowles <klk@google.com.invalid
> >
> > wrote:
> >
> > > Hi all,
> > >
> > > We have a huge Jenkins backlog, surely exacerbated by the fact that our
> > > test time (precommit and postcommit mvn install) has roughly doubled in
> > the
> > > last few days.
> > >
> > > Here's the quick link to the trend:
> > > https://builds.apache.org/view/Beam/job/beam_PostCommit_
> > Java_MavenInstall/
> > > buildTimeTrend
> > >
> > > Good 33m build at 2016-12-09 02:42:
> > > https://builds.apache.org/view/Beam/job/beam_PostCommit_
> > > Java_MavenInstall/2041/
> > >
> > > Bad 59m build at 2016-12-09 18:00 (trigger by timer):
> > > https://builds.apache.org/view/Beam/job/beam_PostCommit_
> > > Java_MavenInstall/2048/
> > >
> > > There are a couple middling runs in between that I can't place
> > immediately
> > > into either bucket. I'm still looking into this, but it is a
> > clicky-clicky
> > > process that hasn't yielded anything yet.
> > >
> > > If anyone knows something, please share your insight.
> > >
> > > Kenn
> > >
> >
>
Re: Jenkins pre/postcommit increased from 35m to 60m+ on Friday
Posted by Kenneth Knowles <kl...@google.com.INVALID>.
Great. That means the timestamp change I made for Travis, ported to
Jenkins, should reveal more.
Meanwhile - any known issues with Jenkins or Maven Central? Status
dashboard for Maven Central doesn't look unhappy.
On Mon, Dec 12, 2016 at 6:25 PM, Dan Halperin <dh...@google.com.invalid>
wrote:
> From the "bad run", the Maven part took 35 minutes and presumably the rest
> is Jenkins / Maven / downloading overhead.
>
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Beam :: Parent .............................. SUCCESS [
> 37.558 s]
> [INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [
> 8.172 s]
> [INFO] Apache Beam :: SDKs ................................ SUCCESS [
> 11.521 s]
> [INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [
> 10.801 s]
> [INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [04:04
> min]
> [INFO] Apache Beam :: Runners ............................. SUCCESS [
> 14.945 s]
> [INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [
> 44.654 s]
> [INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [02:06
> min]
> [INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [
> 33.245 s]
> [INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [
> 4.047 s]
> [INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform
> SUCCESS [04:32 min]
> [INFO] Apache Beam :: SDKs :: Java :: IO :: HDFS .......... SUCCESS [
> 32.009 s]
> [INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [
> 19.006 s]
> [INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [
> 22.021 s]
> [INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [
> 22.817 s]
> [INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [
> 27.276 s]
> [INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [
> 23.662 s]
> [INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [
> 3.115 s]
> [INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter
> SUCCESS [ 17.818 s]
> [INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples
> SUCCESS [ 15.111 s]
> [INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples -
> Java 8 SUCCESS [ 24.477 s]
> [INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [
> 5.759 s]
> [INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library
> SUCCESS [ 22.293 s]
> [INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [
> 31.145 s]
> [INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [
> 8.033 s]
> [INFO] Apache Beam :: Runners :: Flink .................... SUCCESS [
> 6.503 s]
> [INFO] Apache Beam :: Runners :: Flink :: Core ............ SUCCESS [
> 43.593 s]
> [INFO] Apache Beam :: Runners :: Flink :: Examples ........ SUCCESS [
> 20.006 s]
> [INFO] Apache Beam :: Runners :: Spark .................... SUCCESS [03:10
> min]
> [INFO] Apache Beam :: Runners :: Apex ..................... SUCCESS [01:03
> min]
> [INFO] Apache Beam :: Examples ............................ SUCCESS [
> 8.124 s]
> [INFO] Apache Beam :: Examples :: Java .................... SUCCESS [05:29
> min]
> [INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [
> 19.005 s]
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] Total time: 34:30 min
> [INFO] Finished at: 2016-12-09T18:50:49+00:00
> [INFO] Final Memory: 196M/1051M
> [INFO] ------------------------------------------------------------
> ------------
>
>
>
> On Mon, Dec 12, 2016 at 5:36 PM, Kenneth Knowles <kl...@google.com.invalid>
> wrote:
>
> > Hi all,
> >
> > We have a huge Jenkins backlog, surely exacerbated by the fact that our
> > test time (precommit and postcommit mvn install) has roughly doubled in
> the
> > last few days.
> >
> > Here's the quick link to the trend:
> > https://builds.apache.org/view/Beam/job/beam_PostCommit_
> Java_MavenInstall/
> > buildTimeTrend
> >
> > Good 33m build at 2016-12-09 02:42:
> > https://builds.apache.org/view/Beam/job/beam_PostCommit_
> > Java_MavenInstall/2041/
> >
> > Bad 59m build at 2016-12-09 18:00 (trigger by timer):
> > https://builds.apache.org/view/Beam/job/beam_PostCommit_
> > Java_MavenInstall/2048/
> >
> > There are a couple middling runs in between that I can't place
> immediately
> > into either bucket. I'm still looking into this, but it is a
> clicky-clicky
> > process that hasn't yielded anything yet.
> >
> > If anyone knows something, please share your insight.
> >
> > Kenn
> >
>
Re: Jenkins pre/postcommit increased from 35m to 60m+ on Friday
Posted by Dan Halperin <dh...@google.com.INVALID>.
From the "bad run", the Maven part took 35 minutes and presumably the rest
is Jenkins / Maven / downloading overhead.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 37.558 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 8.172 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 11.521 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 10.801 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [04:04 min]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 14.945 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 44.654 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [02:06 min]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 33.245 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 4.047 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform
SUCCESS [04:32 min]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HDFS .......... SUCCESS [ 32.009 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 19.006 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 22.021 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 22.817 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 27.276 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 23.662 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 3.115 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter
SUCCESS [ 17.818 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples
SUCCESS [ 15.111 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples -
Java 8 SUCCESS [ 24.477 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 5.759 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library
SUCCESS [ 22.293 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 31.145 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 8.033 s]
[INFO] Apache Beam :: Runners :: Flink .................... SUCCESS [ 6.503 s]
[INFO] Apache Beam :: Runners :: Flink :: Core ............ SUCCESS [ 43.593 s]
[INFO] Apache Beam :: Runners :: Flink :: Examples ........ SUCCESS [ 20.006 s]
[INFO] Apache Beam :: Runners :: Spark .................... SUCCESS [03:10 min]
[INFO] Apache Beam :: Runners :: Apex ..................... SUCCESS [01:03 min]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 8.124 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [05:29 min]
[INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [ 19.005 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 34:30 min
[INFO] Finished at: 2016-12-09T18:50:49+00:00
[INFO] Final Memory: 196M/1051M
[INFO] ------------------------------------------------------------------------
On Mon, Dec 12, 2016 at 5:36 PM, Kenneth Knowles <kl...@google.com.invalid>
wrote:
> Hi all,
>
> We have a huge Jenkins backlog, surely exacerbated by the fact that our
> test time (precommit and postcommit mvn install) has roughly doubled in the
> last few days.
>
> Here's the quick link to the trend:
> https://builds.apache.org/view/Beam/job/beam_PostCommit_Java_MavenInstall/
> buildTimeTrend
>
> Good 33m build at 2016-12-09 02:42:
> https://builds.apache.org/view/Beam/job/beam_PostCommit_
> Java_MavenInstall/2041/
>
> Bad 59m build at 2016-12-09 18:00 (trigger by timer):
> https://builds.apache.org/view/Beam/job/beam_PostCommit_
> Java_MavenInstall/2048/
>
> There are a couple middling runs in between that I can't place immediately
> into either bucket. I'm still looking into this, but it is a clicky-clicky
> process that hasn't yielded anything yet.
>
> If anyone knows something, please share your insight.
>
> Kenn
>