You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Kenneth Knowles <ke...@apache.org> on 2018/10/23 21:28:33 UTC
Java Precommit duration
Hi all,
Java Precommit duration is about 1h15. That is quite a burden. Especially
if something gets broken. We turned off parallel builds, which we really
need to re-enable. But beyond that, I see low-hanging fruit that would most
appropriately be a separate Jenkins job.
Here's a scan of a successful run:
https://scans.gradle.com/s/2s4bd5hc45wuy/timeline
* 17m :beam-runners-google-cloud-dataflow-java-examples:preCommit
* 4m :beam-runners-google-cloud-dataflow-java-examples-streaming:preCommit
These are integration tests that should have their own job & status anyhow.
We lumped them in because Maven can't do separate tests. Gradle makes this
cheap and easy.
Then there are these which are the only other tasks over 1m:
* 2m :beam-runners-google-cloud-dataflow-java-legacy-worker:test
* 2m :beam-runners-google-cloud-dataflow-java-fn-api-worker:test
* 2m :beam-sdks-java-nexmark:test
* 1m :beam-sdks-java-io-google-cloud-platform:test
* 1m :beam-sdks-java-io-hbase:test
* 1m :beam-sdks-java-extensions-sql:test
Maybe not worth messing with these. Also if we remove all the shadowJar
and shadowTestJar tasks it actually looks like it would only save 5
minutes, so I was hasty in thinking that would solve things. It will make
interactive work better (going from 30s to maybe <10s for rebuilds) but
won't help that much for Jenkins.
Kenn
Re: Java Precommit duration
Posted by Kenneth Knowles <kl...@google.com>.
On Thu, Oct 25, 2018 at 10:47 PM Ruoyun Huang <ru...@google.com> wrote:
> I was trying to reproduce the issue and understand the situation. By
> saying restoring parallel build, does that refer to "org.gradle.parallel"
> in gradle.properties?
>
It is less tricky than that. Here's the exact change:
https://github.com/apache/beam/pull/6838/commits/32897e8ede5062ba7251f82ffdd50a156134c582
Kenn
>
> For me, regardless this gradle parallel property is on or off, running
> javaPreCommit always fails on target "
> *:beam-examples-java:directRunnerPreCommit*". Repo is synced to head.
> What did I miss?
>
>
>
> On Thu, Oct 25, 2018 at 7:57 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> At this point I think the gains would be much less from further
>> splitting. I am looking into parallel build restoration. Is it true that
>> there were primarily the two failures?
>>
>> https://issues.apache.org/jira/browse/BEAM-5207
>> :beam-runners-apex:compileTestJava
>> https://issues.apache.org/jira/browse/BEAM-5035
>> :beam-examples-java:compileTestJava
>>
>> The problem seems extremely likely to be our homebrew shadowTestJar. I
>> looked at it and nothing obvious jumped out at me. But the point should be
>> moot - dependencies like these are simply incorrect. It seems likely to be
>> a (common) misunderstanding of what a test jar is. See
>> https://issues.apache.org/jira/browse/BEAM-3138 and
>> https://issues.apache.org/jira/browse/BEAM-3573. Nothing should ever
>> have a compile dependency on a class in another module's test jar. Runtime
>> is OK for scraping test suites like ValidatesRunner (though this too is
>> fairly fancy and I've never seen other projects do it).
>>
>> So fixing our deps might make the whole problem go away. It looks like as
>> a first step we need to promote some misc. code in the test jar to either
>> the main GCPIO jar or a supplementary _main_ jar of a test-utils module.
>>
>> Kenn
>>
>> On Thu, Oct 25, 2018 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> Perhaps you could further split pre-commits along logical lines:
>>> examples, core, IO, runners, etc.
>>>
>>>
>>> On Thu, Oct 25, 2018 at 4:02 PM Scott Wegner <sc...@apache.org> wrote:
>>>
>>>> Splitting into separate jobs that can be parallelized seems like a win
>>>> for as long as Gradle task parallelization is disabled. Thanks for driving
>>>> this improvement.
>>>>
>>>> > I'm in favor of (simple!) build breaks going in before precommits
>>>> finish, on the promise that the offending test(s) passed locally.
>>>>
>>>> +1. Some of our post-commits run even longer (many hours); it's not
>>>> practical to wait for a full run to merge a fix. For targeted
>>>> fixes/rollbacks to restore Jenkins test signal, I support using a passing
>>>> local run as a stand-in for a full Jenkins execution. Note that if you run
>>>> gradle with the --scan flag, it will produce a Build Scan URL to submit
>>>> with you PR.
>>>>
>>>> On Thu, Oct 25, 2018 at 5:39 AM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>>
>>>>> The split did seemingly trim about 30 minutes off the Java precommit.
>>>>> Of course the difference between 50 and 80 minutes won't qualitatively
>>>>> change much. I don't see any other obvious and easy wins. I still like the
>>>>> split for the separation of signals.
>>>>>
>>>>> Kenn
>>>>>
>>>>> On Tue, Oct 23, 2018 at 2:47 PM Robert Bradshaw <ro...@google.com>
>>>>> wrote:
>>>>>
>>>>>> On Tue, Oct 23, 2018 at 11:28 PM Kenneth Knowles <ke...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Java Precommit duration is about 1h15. That is quite a burden.
>>>>>>> Especially if something gets broken.
>>>>>>>
>>>>>>
>>>>>> I'm in favor of (simple!) build breaks going in before precommits
>>>>>> finish, on the promise that the offending test(s) passed locally. Short of
>>>>>> that, we can roll back.
>>>>>>
>>>>>> If it were cheap to get a fast "this is probably good" signal, that
>>>>>> could be useful as well, though once you hit the "I'm waiting long enough
>>>>>> to go do something else" the difference between 20 minutes and 80 minutes
>>>>>> is not that huge.
>>>>>>
>>>>>>
>>>>>>> We turned off parallel builds, which we really need to re-enable.
>>>>>>>
>>>>>>
>>>>>> +1
>>>>>>
>>>>>>
>>>>>>> But beyond that, I see low-hanging fruit that would most
>>>>>>> appropriately be a separate Jenkins job.
>>>>>>>
>>>>>>> Here's a scan of a successful run:
>>>>>>> https://scans.gradle.com/s/2s4bd5hc45wuy/timeline
>>>>>>>
>>>>>>> * 17m :beam-runners-google-cloud-dataflow-java-examples:preCommit
>>>>>>> * 4m
>>>>>>> :beam-runners-google-cloud-dataflow-java-examples-streaming:preCommit
>>>>>>> These are integration tests that should have their own job & status
>>>>>>> anyhow. We lumped them in because Maven can't do separate tests. Gradle
>>>>>>> makes this cheap and easy.
>>>>>>>
>>>>>>> Then there are these which are the only other tasks over 1m:
>>>>>>>
>>>>>>> * 2m :beam-runners-google-cloud-dataflow-java-legacy-worker:test
>>>>>>> * 2m :beam-runners-google-cloud-dataflow-java-fn-api-worker:test
>>>>>>> * 2m :beam-sdks-java-nexmark:test
>>>>>>> * 1m :beam-sdks-java-io-google-cloud-platform:test
>>>>>>> * 1m :beam-sdks-java-io-hbase:test
>>>>>>> * 1m :beam-sdks-java-extensions-sql:test
>>>>>>>
>>>>>>> Maybe not worth messing with these. Also if we remove all the
>>>>>>> shadowJar and shadowTestJar tasks it actually looks like it would only save
>>>>>>> 5 minutes, so I was hasty in thinking that would solve things. It will make
>>>>>>> interactive work better (going from 30s to maybe <10s for rebuilds) but
>>>>>>> won't help that much for Jenkins.
>>>>>>>
>>>>>>> Kenn
>>>>>>>
>>>>>>
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>>
>>>> Got feedback? tinyurl.com/swegner-feedback
>>>>
>>>
>
> --
> ================
> Ruoyun Huang
>
>
Re: Java Precommit duration
Posted by Ruoyun Huang <ru...@google.com>.
I was trying to reproduce the issue and understand the situation. By
saying restoring parallel build, does that refer to "org.gradle.parallel"
in gradle.properties?
For me, regardless this gradle parallel property is on or off, running
javaPreCommit always fails on target "
*:beam-examples-java:directRunnerPreCommit*". Repo is synced to head.
What did I miss?
On Thu, Oct 25, 2018 at 7:57 PM Kenneth Knowles <ke...@apache.org> wrote:
> At this point I think the gains would be much less from further splitting.
> I am looking into parallel build restoration. Is it true that there were
> primarily the two failures?
>
> https://issues.apache.org/jira/browse/BEAM-5207
> :beam-runners-apex:compileTestJava
> https://issues.apache.org/jira/browse/BEAM-5035
> :beam-examples-java:compileTestJava
>
> The problem seems extremely likely to be our homebrew shadowTestJar. I
> looked at it and nothing obvious jumped out at me. But the point should be
> moot - dependencies like these are simply incorrect. It seems likely to be
> a (common) misunderstanding of what a test jar is. See
> https://issues.apache.org/jira/browse/BEAM-3138 and
> https://issues.apache.org/jira/browse/BEAM-3573. Nothing should ever have
> a compile dependency on a class in another module's test jar. Runtime is OK
> for scraping test suites like ValidatesRunner (though this too is fairly
> fancy and I've never seen other projects do it).
>
> So fixing our deps might make the whole problem go away. It looks like as
> a first step we need to promote some misc. code in the test jar to either
> the main GCPIO jar or a supplementary _main_ jar of a test-utils module.
>
> Kenn
>
> On Thu, Oct 25, 2018 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
>
>> Perhaps you could further split pre-commits along logical lines:
>> examples, core, IO, runners, etc.
>>
>>
>> On Thu, Oct 25, 2018 at 4:02 PM Scott Wegner <sc...@apache.org> wrote:
>>
>>> Splitting into separate jobs that can be parallelized seems like a win
>>> for as long as Gradle task parallelization is disabled. Thanks for driving
>>> this improvement.
>>>
>>> > I'm in favor of (simple!) build breaks going in before precommits
>>> finish, on the promise that the offending test(s) passed locally.
>>>
>>> +1. Some of our post-commits run even longer (many hours); it's not
>>> practical to wait for a full run to merge a fix. For targeted
>>> fixes/rollbacks to restore Jenkins test signal, I support using a passing
>>> local run as a stand-in for a full Jenkins execution. Note that if you run
>>> gradle with the --scan flag, it will produce a Build Scan URL to submit
>>> with you PR.
>>>
>>> On Thu, Oct 25, 2018 at 5:39 AM Kenneth Knowles <ke...@apache.org> wrote:
>>>
>>>> The split did seemingly trim about 30 minutes off the Java precommit.
>>>> Of course the difference between 50 and 80 minutes won't qualitatively
>>>> change much. I don't see any other obvious and easy wins. I still like the
>>>> split for the separation of signals.
>>>>
>>>> Kenn
>>>>
>>>> On Tue, Oct 23, 2018 at 2:47 PM Robert Bradshaw <ro...@google.com>
>>>> wrote:
>>>>
>>>>> On Tue, Oct 23, 2018 at 11:28 PM Kenneth Knowles <ke...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Java Precommit duration is about 1h15. That is quite a burden.
>>>>>> Especially if something gets broken.
>>>>>>
>>>>>
>>>>> I'm in favor of (simple!) build breaks going in before precommits
>>>>> finish, on the promise that the offending test(s) passed locally. Short of
>>>>> that, we can roll back.
>>>>>
>>>>> If it were cheap to get a fast "this is probably good" signal, that
>>>>> could be useful as well, though once you hit the "I'm waiting long enough
>>>>> to go do something else" the difference between 20 minutes and 80 minutes
>>>>> is not that huge.
>>>>>
>>>>>
>>>>>> We turned off parallel builds, which we really need to re-enable.
>>>>>>
>>>>>
>>>>> +1
>>>>>
>>>>>
>>>>>> But beyond that, I see low-hanging fruit that would most
>>>>>> appropriately be a separate Jenkins job.
>>>>>>
>>>>>> Here's a scan of a successful run:
>>>>>> https://scans.gradle.com/s/2s4bd5hc45wuy/timeline
>>>>>>
>>>>>> * 17m :beam-runners-google-cloud-dataflow-java-examples:preCommit
>>>>>> * 4m
>>>>>> :beam-runners-google-cloud-dataflow-java-examples-streaming:preCommit
>>>>>> These are integration tests that should have their own job & status
>>>>>> anyhow. We lumped them in because Maven can't do separate tests. Gradle
>>>>>> makes this cheap and easy.
>>>>>>
>>>>>> Then there are these which are the only other tasks over 1m:
>>>>>>
>>>>>> * 2m :beam-runners-google-cloud-dataflow-java-legacy-worker:test
>>>>>> * 2m :beam-runners-google-cloud-dataflow-java-fn-api-worker:test
>>>>>> * 2m :beam-sdks-java-nexmark:test
>>>>>> * 1m :beam-sdks-java-io-google-cloud-platform:test
>>>>>> * 1m :beam-sdks-java-io-hbase:test
>>>>>> * 1m :beam-sdks-java-extensions-sql:test
>>>>>>
>>>>>> Maybe not worth messing with these. Also if we remove all the
>>>>>> shadowJar and shadowTestJar tasks it actually looks like it would only save
>>>>>> 5 minutes, so I was hasty in thinking that would solve things. It will make
>>>>>> interactive work better (going from 30s to maybe <10s for rebuilds) but
>>>>>> won't help that much for Jenkins.
>>>>>>
>>>>>> Kenn
>>>>>>
>>>>>
>>>
>>> --
>>>
>>>
>>>
>>>
>>> Got feedback? tinyurl.com/swegner-feedback
>>>
>>
--
================
Ruoyun Huang
Re: Java Precommit duration
Posted by Kenneth Knowles <ke...@apache.org>.
At this point I think the gains would be much less from further splitting.
I am looking into parallel build restoration. Is it true that there were
primarily the two failures?
https://issues.apache.org/jira/browse/BEAM-5207
:beam-runners-apex:compileTestJava
https://issues.apache.org/jira/browse/BEAM-5035
:beam-examples-java:compileTestJava
The problem seems extremely likely to be our homebrew shadowTestJar. I
looked at it and nothing obvious jumped out at me. But the point should be
moot - dependencies like these are simply incorrect. It seems likely to be
a (common) misunderstanding of what a test jar is. See
https://issues.apache.org/jira/browse/BEAM-3138 and
https://issues.apache.org/jira/browse/BEAM-3573. Nothing should ever have a
compile dependency on a class in another module's test jar. Runtime is OK
for scraping test suites like ValidatesRunner (though this too is fairly
fancy and I've never seen other projects do it).
So fixing our deps might make the whole problem go away. It looks like as a
first step we need to promote some misc. code in the test jar to either the
main GCPIO jar or a supplementary _main_ jar of a test-utils module.
Kenn
On Thu, Oct 25, 2018 at 4:21 PM Udi Meiri <eh...@google.com> wrote:
> Perhaps you could further split pre-commits along logical lines: examples,
> core, IO, runners, etc.
>
>
> On Thu, Oct 25, 2018 at 4:02 PM Scott Wegner <sc...@apache.org> wrote:
>
>> Splitting into separate jobs that can be parallelized seems like a win
>> for as long as Gradle task parallelization is disabled. Thanks for driving
>> this improvement.
>>
>> > I'm in favor of (simple!) build breaks going in before precommits
>> finish, on the promise that the offending test(s) passed locally.
>>
>> +1. Some of our post-commits run even longer (many hours); it's not
>> practical to wait for a full run to merge a fix. For targeted
>> fixes/rollbacks to restore Jenkins test signal, I support using a passing
>> local run as a stand-in for a full Jenkins execution. Note that if you run
>> gradle with the --scan flag, it will produce a Build Scan URL to submit
>> with you PR.
>>
>> On Thu, Oct 25, 2018 at 5:39 AM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> The split did seemingly trim about 30 minutes off the Java precommit. Of
>>> course the difference between 50 and 80 minutes won't qualitatively change
>>> much. I don't see any other obvious and easy wins. I still like the split
>>> for the separation of signals.
>>>
>>> Kenn
>>>
>>> On Tue, Oct 23, 2018 at 2:47 PM Robert Bradshaw <ro...@google.com>
>>> wrote:
>>>
>>>> On Tue, Oct 23, 2018 at 11:28 PM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Java Precommit duration is about 1h15. That is quite a burden.
>>>>> Especially if something gets broken.
>>>>>
>>>>
>>>> I'm in favor of (simple!) build breaks going in before precommits
>>>> finish, on the promise that the offending test(s) passed locally. Short of
>>>> that, we can roll back.
>>>>
>>>> If it were cheap to get a fast "this is probably good" signal, that
>>>> could be useful as well, though once you hit the "I'm waiting long enough
>>>> to go do something else" the difference between 20 minutes and 80 minutes
>>>> is not that huge.
>>>>
>>>>
>>>>> We turned off parallel builds, which we really need to re-enable.
>>>>>
>>>>
>>>> +1
>>>>
>>>>
>>>>> But beyond that, I see low-hanging fruit that would most appropriately
>>>>> be a separate Jenkins job.
>>>>>
>>>>> Here's a scan of a successful run:
>>>>> https://scans.gradle.com/s/2s4bd5hc45wuy/timeline
>>>>>
>>>>> * 17m :beam-runners-google-cloud-dataflow-java-examples:preCommit
>>>>> * 4m
>>>>> :beam-runners-google-cloud-dataflow-java-examples-streaming:preCommit
>>>>> These are integration tests that should have their own job & status
>>>>> anyhow. We lumped them in because Maven can't do separate tests. Gradle
>>>>> makes this cheap and easy.
>>>>>
>>>>> Then there are these which are the only other tasks over 1m:
>>>>>
>>>>> * 2m :beam-runners-google-cloud-dataflow-java-legacy-worker:test
>>>>> * 2m :beam-runners-google-cloud-dataflow-java-fn-api-worker:test
>>>>> * 2m :beam-sdks-java-nexmark:test
>>>>> * 1m :beam-sdks-java-io-google-cloud-platform:test
>>>>> * 1m :beam-sdks-java-io-hbase:test
>>>>> * 1m :beam-sdks-java-extensions-sql:test
>>>>>
>>>>> Maybe not worth messing with these. Also if we remove all the
>>>>> shadowJar and shadowTestJar tasks it actually looks like it would only save
>>>>> 5 minutes, so I was hasty in thinking that would solve things. It will make
>>>>> interactive work better (going from 30s to maybe <10s for rebuilds) but
>>>>> won't help that much for Jenkins.
>>>>>
>>>>> Kenn
>>>>>
>>>>
>>
>> --
>>
>>
>>
>>
>> Got feedback? tinyurl.com/swegner-feedback
>>
>
Re: Java Precommit duration
Posted by Udi Meiri <eh...@google.com>.
Perhaps you could further split pre-commits along logical lines: examples,
core, IO, runners, etc.
On Thu, Oct 25, 2018 at 4:02 PM Scott Wegner <sc...@apache.org> wrote:
> Splitting into separate jobs that can be parallelized seems like a win for
> as long as Gradle task parallelization is disabled. Thanks for driving this
> improvement.
>
> > I'm in favor of (simple!) build breaks going in before precommits
> finish, on the promise that the offending test(s) passed locally.
>
> +1. Some of our post-commits run even longer (many hours); it's not
> practical to wait for a full run to merge a fix. For targeted
> fixes/rollbacks to restore Jenkins test signal, I support using a passing
> local run as a stand-in for a full Jenkins execution. Note that if you run
> gradle with the --scan flag, it will produce a Build Scan URL to submit
> with you PR.
>
> On Thu, Oct 25, 2018 at 5:39 AM Kenneth Knowles <ke...@apache.org> wrote:
>
>> The split did seemingly trim about 30 minutes off the Java precommit. Of
>> course the difference between 50 and 80 minutes won't qualitatively change
>> much. I don't see any other obvious and easy wins. I still like the split
>> for the separation of signals.
>>
>> Kenn
>>
>> On Tue, Oct 23, 2018 at 2:47 PM Robert Bradshaw <ro...@google.com>
>> wrote:
>>
>>> On Tue, Oct 23, 2018 at 11:28 PM Kenneth Knowles <ke...@apache.org>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Java Precommit duration is about 1h15. That is quite a burden.
>>>> Especially if something gets broken.
>>>>
>>>
>>> I'm in favor of (simple!) build breaks going in before precommits
>>> finish, on the promise that the offending test(s) passed locally. Short of
>>> that, we can roll back.
>>>
>>> If it were cheap to get a fast "this is probably good" signal, that
>>> could be useful as well, though once you hit the "I'm waiting long enough
>>> to go do something else" the difference between 20 minutes and 80 minutes
>>> is not that huge.
>>>
>>>
>>>> We turned off parallel builds, which we really need to re-enable.
>>>>
>>>
>>> +1
>>>
>>>
>>>> But beyond that, I see low-hanging fruit that would most appropriately
>>>> be a separate Jenkins job.
>>>>
>>>> Here's a scan of a successful run:
>>>> https://scans.gradle.com/s/2s4bd5hc45wuy/timeline
>>>>
>>>> * 17m :beam-runners-google-cloud-dataflow-java-examples:preCommit
>>>> * 4m
>>>> :beam-runners-google-cloud-dataflow-java-examples-streaming:preCommit
>>>> These are integration tests that should have their own job & status
>>>> anyhow. We lumped them in because Maven can't do separate tests. Gradle
>>>> makes this cheap and easy.
>>>>
>>>> Then there are these which are the only other tasks over 1m:
>>>>
>>>> * 2m :beam-runners-google-cloud-dataflow-java-legacy-worker:test
>>>> * 2m :beam-runners-google-cloud-dataflow-java-fn-api-worker:test
>>>> * 2m :beam-sdks-java-nexmark:test
>>>> * 1m :beam-sdks-java-io-google-cloud-platform:test
>>>> * 1m :beam-sdks-java-io-hbase:test
>>>> * 1m :beam-sdks-java-extensions-sql:test
>>>>
>>>> Maybe not worth messing with these. Also if we remove all the
>>>> shadowJar and shadowTestJar tasks it actually looks like it would only save
>>>> 5 minutes, so I was hasty in thinking that would solve things. It will make
>>>> interactive work better (going from 30s to maybe <10s for rebuilds) but
>>>> won't help that much for Jenkins.
>>>>
>>>> Kenn
>>>>
>>>
>
> --
>
>
>
>
> Got feedback? tinyurl.com/swegner-feedback
>
Re: Java Precommit duration
Posted by Scott Wegner <sc...@apache.org>.
Splitting into separate jobs that can be parallelized seems like a win for
as long as Gradle task parallelization is disabled. Thanks for driving this
improvement.
> I'm in favor of (simple!) build breaks going in before precommits finish,
on the promise that the offending test(s) passed locally.
+1. Some of our post-commits run even longer (many hours); it's not
practical to wait for a full run to merge a fix. For targeted
fixes/rollbacks to restore Jenkins test signal, I support using a passing
local run as a stand-in for a full Jenkins execution. Note that if you run
gradle with the --scan flag, it will produce a Build Scan URL to submit
with you PR.
On Thu, Oct 25, 2018 at 5:39 AM Kenneth Knowles <ke...@apache.org> wrote:
> The split did seemingly trim about 30 minutes off the Java precommit. Of
> course the difference between 50 and 80 minutes won't qualitatively change
> much. I don't see any other obvious and easy wins. I still like the split
> for the separation of signals.
>
> Kenn
>
> On Tue, Oct 23, 2018 at 2:47 PM Robert Bradshaw <ro...@google.com>
> wrote:
>
>> On Tue, Oct 23, 2018 at 11:28 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> Hi all,
>>>
>>> Java Precommit duration is about 1h15. That is quite a burden.
>>> Especially if something gets broken.
>>>
>>
>> I'm in favor of (simple!) build breaks going in before precommits finish,
>> on the promise that the offending test(s) passed locally. Short of that, we
>> can roll back.
>>
>> If it were cheap to get a fast "this is probably good" signal, that could
>> be useful as well, though once you hit the "I'm waiting long enough to go
>> do something else" the difference between 20 minutes and 80 minutes is not
>> that huge.
>>
>>
>>> We turned off parallel builds, which we really need to re-enable.
>>>
>>
>> +1
>>
>>
>>> But beyond that, I see low-hanging fruit that would most appropriately
>>> be a separate Jenkins job.
>>>
>>> Here's a scan of a successful run:
>>> https://scans.gradle.com/s/2s4bd5hc45wuy/timeline
>>>
>>> * 17m :beam-runners-google-cloud-dataflow-java-examples:preCommit
>>> * 4m
>>> :beam-runners-google-cloud-dataflow-java-examples-streaming:preCommit
>>> These are integration tests that should have their own job & status
>>> anyhow. We lumped them in because Maven can't do separate tests. Gradle
>>> makes this cheap and easy.
>>>
>>> Then there are these which are the only other tasks over 1m:
>>>
>>> * 2m :beam-runners-google-cloud-dataflow-java-legacy-worker:test
>>> * 2m :beam-runners-google-cloud-dataflow-java-fn-api-worker:test
>>> * 2m :beam-sdks-java-nexmark:test
>>> * 1m :beam-sdks-java-io-google-cloud-platform:test
>>> * 1m :beam-sdks-java-io-hbase:test
>>> * 1m :beam-sdks-java-extensions-sql:test
>>>
>>> Maybe not worth messing with these. Also if we remove all the shadowJar
>>> and shadowTestJar tasks it actually looks like it would only save 5
>>> minutes, so I was hasty in thinking that would solve things. It will make
>>> interactive work better (going from 30s to maybe <10s for rebuilds) but
>>> won't help that much for Jenkins.
>>>
>>> Kenn
>>>
>>
--
Got feedback? tinyurl.com/swegner-feedback
Re: Java Precommit duration
Posted by Kenneth Knowles <ke...@apache.org>.
The split did seemingly trim about 30 minutes off the Java precommit. Of
course the difference between 50 and 80 minutes won't qualitatively change
much. I don't see any other obvious and easy wins. I still like the split
for the separation of signals.
Kenn
On Tue, Oct 23, 2018 at 2:47 PM Robert Bradshaw <ro...@google.com> wrote:
> On Tue, Oct 23, 2018 at 11:28 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> Hi all,
>>
>> Java Precommit duration is about 1h15. That is quite a burden. Especially
>> if something gets broken.
>>
>
> I'm in favor of (simple!) build breaks going in before precommits finish,
> on the promise that the offending test(s) passed locally. Short of that, we
> can roll back.
>
> If it were cheap to get a fast "this is probably good" signal, that could
> be useful as well, though once you hit the "I'm waiting long enough to go
> do something else" the difference between 20 minutes and 80 minutes is not
> that huge.
>
>
>> We turned off parallel builds, which we really need to re-enable.
>>
>
> +1
>
>
>> But beyond that, I see low-hanging fruit that would most appropriately be
>> a separate Jenkins job.
>>
>> Here's a scan of a successful run:
>> https://scans.gradle.com/s/2s4bd5hc45wuy/timeline
>>
>> * 17m :beam-runners-google-cloud-dataflow-java-examples:preCommit
>> * 4m :beam-runners-google-cloud-dataflow-java-examples-streaming:preCommit
>> These are integration tests that should have their own job & status
>> anyhow. We lumped them in because Maven can't do separate tests. Gradle
>> makes this cheap and easy.
>>
>> Then there are these which are the only other tasks over 1m:
>>
>> * 2m :beam-runners-google-cloud-dataflow-java-legacy-worker:test
>> * 2m :beam-runners-google-cloud-dataflow-java-fn-api-worker:test
>> * 2m :beam-sdks-java-nexmark:test
>> * 1m :beam-sdks-java-io-google-cloud-platform:test
>> * 1m :beam-sdks-java-io-hbase:test
>> * 1m :beam-sdks-java-extensions-sql:test
>>
>> Maybe not worth messing with these. Also if we remove all the shadowJar
>> and shadowTestJar tasks it actually looks like it would only save 5
>> minutes, so I was hasty in thinking that would solve things. It will make
>> interactive work better (going from 30s to maybe <10s for rebuilds) but
>> won't help that much for Jenkins.
>>
>> Kenn
>>
>
Re: Java Precommit duration
Posted by Robert Bradshaw <ro...@google.com>.
On Tue, Oct 23, 2018 at 11:28 PM Kenneth Knowles <ke...@apache.org> wrote:
> Hi all,
>
> Java Precommit duration is about 1h15. That is quite a burden. Especially
> if something gets broken.
>
I'm in favor of (simple!) build breaks going in before precommits finish,
on the promise that the offending test(s) passed locally. Short of that, we
can roll back.
If it were cheap to get a fast "this is probably good" signal, that could
be useful as well, though once you hit the "I'm waiting long enough to go
do something else" the difference between 20 minutes and 80 minutes is not
that huge.
> We turned off parallel builds, which we really need to re-enable.
>
+1
> But beyond that, I see low-hanging fruit that would most appropriately be
> a separate Jenkins job.
>
> Here's a scan of a successful run:
> https://scans.gradle.com/s/2s4bd5hc45wuy/timeline
>
> * 17m :beam-runners-google-cloud-dataflow-java-examples:preCommit
> * 4m :beam-runners-google-cloud-dataflow-java-examples-streaming:preCommit
> These are integration tests that should have their own job & status
> anyhow. We lumped them in because Maven can't do separate tests. Gradle
> makes this cheap and easy.
>
> Then there are these which are the only other tasks over 1m:
>
> * 2m :beam-runners-google-cloud-dataflow-java-legacy-worker:test
> * 2m :beam-runners-google-cloud-dataflow-java-fn-api-worker:test
> * 2m :beam-sdks-java-nexmark:test
> * 1m :beam-sdks-java-io-google-cloud-platform:test
> * 1m :beam-sdks-java-io-hbase:test
> * 1m :beam-sdks-java-extensions-sql:test
>
> Maybe not worth messing with these. Also if we remove all the shadowJar
> and shadowTestJar tasks it actually looks like it would only save 5
> minutes, so I was hasty in thinking that would solve things. It will make
> interactive work better (going from 30s to maybe <10s for rebuilds) but
> won't help that much for Jenkins.
>
> Kenn
>