Re: Questions about TeamCity CI organization

[Dmitriy Setrakyan <ds...@apache.org>]

I am also curious to get answers to the questions posted by Raul. Can
someone knowledgeable in TC configuration respond?

D.

On Mon, Oct 26, 2015 at 6:14 AM, Raul Kripalani <ra...@apache.org> wrote:

> Hey guys,
>
> I have a few questions about the way our builds are organised in TeamCity.
>
> 1. All build configurations seem to be building the entire project tree. Is
> this correct?
>
> 2. The goal of the build configurations here is to partition the tests so
> that the test execution can be distributed across all 15 worker nodes,
> right?
>
> 3. Would it make sense to have a single top-level build for the project
> tree that actually builds the code? This one would in turn trigger the
> individual test suites – each in a build configuration such that then can
> continue to be distributed across the worker cloud? Or do we face a problem
> with Maven SNAPSHOTs potentially overwriting each other if multiple builds
> are running concurrently? To me, it seems overkill to build the entire
> project tree over and over again only to execute a given test suite (if I
> understood correctly).
>
> 4. Are tests for ignite-spark, ignite-kafka, ignite-mesos,
> ignite-rest-http, etc. ever run? I don't find specific build configurations
> for them... Are they included in other configs?
>
> Thanks,
>
> *Raúl Kripalani*
> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> Messaging Engineer
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk
>

Re: Questions about TeamCity CI organization

[Artem Shutak <as...@gridgain.com>]

Hi Raul,

I agree, we need to have the wiki page. Could you please initiate the one
with topic which should be covered and provide a link? I will help with
filling.

Thanks,
-- Artem --

On Tue, Nov 3, 2015 at 8:02 PM, Raul Kripalani <ra...@apache.org> wrote:

> Hi Artem,
>
> Thanks for the clarifications.
>
> At first, it did seem overkill that we're building the code over and over
> again in each build configuration – when the ultimate goal is just to run a
> subset of tests.
>
> But you are right in that the build itself takes very little time
> comparatively to the tests. Separating the build on one side (single build
> config) and tests on the other (1 build config per test group) can be
> difficult and risky because it's hard to guarantee that tests distributed
> for execution across a cluster will run against exactly the same SNAPSHOT
> built by the top-level job.
>
> Logically it would be more efficient, but practically it would be to
> over-optimise in the current context.
>
> BTW – We need to add TC management to the Ignite Wiki. In particular, the
> committer merging a new module (or Test Suite) should ensure that it's
> covered by a TC job.
>
> Thanks,
>
> *Raúl Kripalani*
> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> Messaging Engineer
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk
>
> On Tue, Nov 3, 2015 at 12:05 PM, Artem Shutak <as...@gridgain.com>
> wrote:
>
> > Hi Raul,
> >
> > 1. Yes, this is correct separation. If you are working on some new
> feature
> > usually in development process you want to run only your build, may be 2,
> > but not all tests. And only when a feature is close to complete you want
> to
> > run all tests.
> >
> > 2. There are at least 2 goals:
> > - Don't run all tests in process of development - see point 1
> > - the test execution can be distributed across many nodes
> >
> > 3. I don't see any issue here. At first, we use a standard maven test
> > procedure. At second, a test execution for some tasks (cache for example)
> > takes 30-60 minutes, maven build takes 2 minutes (from 30-60 min).
> > And yes, there're lots of problems which will need to solve to escape a
> > rebuilding a project each time but it will not give any speed up.
> >
> > 4. Spark, Kafka, Mesos - new modules. Tests for the modules were not
> added
> > by mistake. I think a person who commit new module should add a
> > corresponding test plan to TC.
> > I don't see tests into 'rest-http' module. As I understand this
> > functionality covered by tests in another modules.
> >
> > Do you have some proposals about CI organization? I think there is no
> needs
> > to change something.
> >
> > Thanks,
> > -- Artem --
> >
> > On Tue, Nov 3, 2015 at 12:27 AM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> >
> > > I am also curious to get answers to the questions posted by Raul. Can
> > > someone knowledgeable in TC configuration respond?
> > >
> > > D.
> > >
> > > On Mon, Oct 26, 2015 at 6:14 AM, Raul Kripalani <ra...@apache.org>
> > wrote:
> > >
> > > > Hey guys,
> > > >
> > > > I have a few questions about the way our builds are organised in
> > > TeamCity.
> > > >
> > > > 1. All build configurations seem to be building the entire project
> > tree.
> > > Is
> > > > this correct?
> > > >
> > > > 2. The goal of the build configurations here is to partition the
> tests
> > so
> > > > that the test execution can be distributed across all 15 worker
> nodes,
> > > > right?
> > > >
> > > > 3. Would it make sense to have a single top-level build for the
> project
> > > > tree that actually builds the code? This one would in turn trigger
> the
> > > > individual test suites – each in a build configuration such that then
> > can
> > > > continue to be distributed across the worker cloud? Or do we face a
> > > problem
> > > > with Maven SNAPSHOTs potentially overwriting each other if multiple
> > > builds
> > > > are running concurrently? To me, it seems overkill to build the
> entire
> > > > project tree over and over again only to execute a given test suite
> > (if I
> > > > understood correctly).
> > > >
> > > > 4. Are tests for ignite-spark, ignite-kafka, ignite-mesos,
> > > > ignite-rest-http, etc. ever run? I don't find specific build
> > > configurations
> > > > for them... Are they included in other configs?
> > > >
> > > > Thanks,
> > > >
> > > > *Raúl Kripalani*
> > > > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data
> > and
> > > > Messaging Engineer
> > > > http://about.me/raulkripalani |
> > http://www.linkedin.com/in/raulkripalani
> > > > http://blog.raulkr.net | twitter: @raulvk
> > > >
> > >
> >
>

Re: Questions about TeamCity CI organization

[Raul Kripalani <ra...@apache.org>]

Hi Artem,

Thanks for the clarifications.

At first, it did seem overkill that we're building the code over and over
again in each build configuration – when the ultimate goal is just to run a
subset of tests.

But you are right in that the build itself takes very little time
comparatively to the tests. Separating the build on one side (single build
config) and tests on the other (1 build config per test group) can be
difficult and risky because it's hard to guarantee that tests distributed
for execution across a cluster will run against exactly the same SNAPSHOT
built by the top-level job.

Logically it would be more efficient, but practically it would be to
over-optimise in the current context.

BTW – We need to add TC management to the Ignite Wiki. In particular, the
committer merging a new module (or Test Suite) should ensure that it's
covered by a TC job.

Thanks,

*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk

On Tue, Nov 3, 2015 at 12:05 PM, Artem Shutak <as...@gridgain.com> wrote:

> Hi Raul,
>
> 1. Yes, this is correct separation. If you are working on some new feature
> usually in development process you want to run only your build, may be 2,
> but not all tests. And only when a feature is close to complete you want to
> run all tests.
>
> 2. There are at least 2 goals:
> - Don't run all tests in process of development - see point 1
> - the test execution can be distributed across many nodes
>
> 3. I don't see any issue here. At first, we use a standard maven test
> procedure. At second, a test execution for some tasks (cache for example)
> takes 30-60 minutes, maven build takes 2 minutes (from 30-60 min).
> And yes, there're lots of problems which will need to solve to escape a
> rebuilding a project each time but it will not give any speed up.
>
> 4. Spark, Kafka, Mesos - new modules. Tests for the modules were not added
> by mistake. I think a person who commit new module should add a
> corresponding test plan to TC.
> I don't see tests into 'rest-http' module. As I understand this
> functionality covered by tests in another modules.
>
> Do you have some proposals about CI organization? I think there is no needs
> to change something.
>
> Thanks,
> -- Artem --
>
> On Tue, Nov 3, 2015 at 12:27 AM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > I am also curious to get answers to the questions posted by Raul. Can
> > someone knowledgeable in TC configuration respond?
> >
> > D.
> >
> > On Mon, Oct 26, 2015 at 6:14 AM, Raul Kripalani <ra...@apache.org>
> wrote:
> >
> > > Hey guys,
> > >
> > > I have a few questions about the way our builds are organised in
> > TeamCity.
> > >
> > > 1. All build configurations seem to be building the entire project
> tree.
> > Is
> > > this correct?
> > >
> > > 2. The goal of the build configurations here is to partition the tests
> so
> > > that the test execution can be distributed across all 15 worker nodes,
> > > right?
> > >
> > > 3. Would it make sense to have a single top-level build for the project
> > > tree that actually builds the code? This one would in turn trigger the
> > > individual test suites – each in a build configuration such that then
> can
> > > continue to be distributed across the worker cloud? Or do we face a
> > problem
> > > with Maven SNAPSHOTs potentially overwriting each other if multiple
> > builds
> > > are running concurrently? To me, it seems overkill to build the entire
> > > project tree over and over again only to execute a given test suite
> (if I
> > > understood correctly).
> > >
> > > 4. Are tests for ignite-spark, ignite-kafka, ignite-mesos,
> > > ignite-rest-http, etc. ever run? I don't find specific build
> > configurations
> > > for them... Are they included in other configs?
> > >
> > > Thanks,
> > >
> > > *Raúl Kripalani*
> > > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data
> and
> > > Messaging Engineer
> > > http://about.me/raulkripalani |
> http://www.linkedin.com/in/raulkripalani
> > > http://blog.raulkr.net | twitter: @raulvk
> > >
> >
>

Re: Questions about TeamCity CI organization

[Artem Shutak <as...@gridgain.com>]

Hi Raul,

1. Yes, this is correct separation. If you are working on some new feature
usually in development process you want to run only your build, may be 2,
but not all tests. And only when a feature is close to complete you want to
run all tests.

2. There are at least 2 goals:
- Don't run all tests in process of development - see point 1
- the test execution can be distributed across many nodes

3. I don't see any issue here. At first, we use a standard maven test
procedure. At second, a test execution for some tasks (cache for example)
takes 30-60 minutes, maven build takes 2 minutes (from 30-60 min).
And yes, there're lots of problems which will need to solve to escape a
rebuilding a project each time but it will not give any speed up.

4. Spark, Kafka, Mesos - new modules. Tests for the modules were not added
by mistake. I think a person who commit new module should add a
corresponding test plan to TC.
I don't see tests into 'rest-http' module. As I understand this
functionality covered by tests in another modules.

Do you have some proposals about CI organization? I think there is no needs
to change something.

Thanks,
-- Artem --

On Tue, Nov 3, 2015 at 12:27 AM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> I am also curious to get answers to the questions posted by Raul. Can
> someone knowledgeable in TC configuration respond?
>
> D.
>
> On Mon, Oct 26, 2015 at 6:14 AM, Raul Kripalani <ra...@apache.org> wrote:
>
> > Hey guys,
> >
> > I have a few questions about the way our builds are organised in
> TeamCity.
> >
> > 1. All build configurations seem to be building the entire project tree.
> Is
> > this correct?
> >
> > 2. The goal of the build configurations here is to partition the tests so
> > that the test execution can be distributed across all 15 worker nodes,
> > right?
> >
> > 3. Would it make sense to have a single top-level build for the project
> > tree that actually builds the code? This one would in turn trigger the
> > individual test suites – each in a build configuration such that then can
> > continue to be distributed across the worker cloud? Or do we face a
> problem
> > with Maven SNAPSHOTs potentially overwriting each other if multiple
> builds
> > are running concurrently? To me, it seems overkill to build the entire
> > project tree over and over again only to execute a given test suite (if I
> > understood correctly).
> >
> > 4. Are tests for ignite-spark, ignite-kafka, ignite-mesos,
> > ignite-rest-http, etc. ever run? I don't find specific build
> configurations
> > for them... Are they included in other configs?
> >
> > Thanks,
> >
> > *Raúl Kripalani*
> > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> > Messaging Engineer
> > http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> > http://blog.raulkr.net | twitter: @raulvk
> >
>