You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Łukasz Gajowy <lu...@gmail.com> on 2018/04/16 14:48:39 UTC
Rebuilding beam before running Performance Test - question.
Hi all,
currently performance tests running on Jenkins all have "beam_prebuilt" [1]
Perfkit's flag set to true, which means that PerfKit does not rebuild the
code before invoking the Performance Test. This makes things faster but
error prone - we observed Performance Tests failures several times due to
the fact that something was not built on time.
We should rebuild Beam in every testing job to avoid errors (only "bare"
build, without tests and checkstyle). This will make the tests last longer
(about 7 minutes per each test, as my experiments have shown). Probably it
will be faster on Gradle (didn't test it yet). There are 12 tests now with
"beam_PerformanceTests_JDBC" as the longest lasting (total of 15 minutes).
It may be just a "formality question", but does anyone object that we
rebuild beam like this in each Performance Tests Jenkins job?
[1]
https://beam.apache.org/documentation/io/testing/#implementing-integration-tests
Best regards,
Łukasz
Re: Rebuilding beam before running Performance Test - question.
Posted by Łukasz Gajowy <lu...@gmail.com>.
ok, fyi I created a PR for this here:
https://github.com/apache/beam/pull/5153
2018-04-16 19:40 GMT+02:00 Jason Kuster <ja...@google.com>:
> I think that should be fine; I believe the way it was that way originally
> was because we already had things set up for building in Jenkins, but it
> was a while ago. Given the performance tests run infrequently enough, I
> don't think an increase in runtime will be a big deal.
>
> On Mon, Apr 16, 2018 at 7:48 AM Łukasz Gajowy <lu...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> currently performance tests running on Jenkins all have "beam_prebuilt"
>> [1] Perfkit's flag set to true, which means that PerfKit does not rebuild
>> the code before invoking the Performance Test. This makes things faster but
>> error prone - we observed Performance Tests failures several times due to
>> the fact that something was not built on time.
>>
>> We should rebuild Beam in every testing job to avoid errors (only "bare"
>> build, without tests and checkstyle). This will make the tests last longer
>> (about 7 minutes per each test, as my experiments have shown). Probably it
>> will be faster on Gradle (didn't test it yet). There are 12 tests now with
>> "beam_PerformanceTests_JDBC" as the longest lasting (total of 15 minutes).
>>
>> It may be just a "formality question", but does anyone object that we
>> rebuild beam like this in each Performance Tests Jenkins job?
>>
>> [1] https://beam.apache.org/documentation/io/testing/#
>> implementing-integration-tests
>>
>> Best regards,
>> Łukasz
>>
>
>
> --
> -------
> Jason Kuster
> Apache Beam / Google Cloud Dataflow
>
> See something? Say something. go/jasonkuster-feedback
>
Re: Rebuilding beam before running Performance Test - question.
Posted by Jason Kuster <ja...@google.com>.
I think that should be fine; I believe the way it was that way originally
was because we already had things set up for building in Jenkins, but it
was a while ago. Given the performance tests run infrequently enough, I
don't think an increase in runtime will be a big deal.
On Mon, Apr 16, 2018 at 7:48 AM Łukasz Gajowy <lu...@gmail.com>
wrote:
> Hi all,
>
> currently performance tests running on Jenkins all have "beam_prebuilt"
> [1] Perfkit's flag set to true, which means that PerfKit does not rebuild
> the code before invoking the Performance Test. This makes things faster but
> error prone - we observed Performance Tests failures several times due to
> the fact that something was not built on time.
>
> We should rebuild Beam in every testing job to avoid errors (only "bare"
> build, without tests and checkstyle). This will make the tests last longer
> (about 7 minutes per each test, as my experiments have shown). Probably it
> will be faster on Gradle (didn't test it yet). There are 12 tests now with
> "beam_PerformanceTests_JDBC" as the longest lasting (total of 15 minutes).
>
> It may be just a "formality question", but does anyone object that we
> rebuild beam like this in each Performance Tests Jenkins job?
>
> [1]
> https://beam.apache.org/documentation/io/testing/#implementing-integration-tests
>
> Best regards,
> Łukasz
>
--
-------
Jason Kuster
Apache Beam / Google Cloud Dataflow
See something? Say something. go/jasonkuster-feedback