Build dependencies issue with 1.4.1

[Philippe Busque <pb...@mediagrif.com>]

Hello,
In January, we upgraded Continuum from version 1.3.8 to 1.4.1.  This helped solve one major issue we had with Continuum: speed due to permission pollution.

However, we've been getting more and more report from our developer than their change weren't in the daily build.  After investigation, we noticed that the build queue has changed since 1.3.8.

We have many project group. Each group contains a series of project that are dependant to each other.
In 1.3.8,  when a scheduled build was triggered, Continuum would process each group, render the dependency and enqueue the builds in a unique queue per group.
So group A would go in queue 1, group B in queue 2, group C in queue 1 and so on. We had 6 build queues and everything was built in order.

Since 1.4.1, we noticed the project were build out of order, more likely due to the fact that various project in groups were build into different queue. So project  A that depend on B who depend on C could be compiled with an older copy of C because it got queued after B and B compiled much faster than C.

Is there a way to restore the old way for queueing behaviour? Or enable parallel build based on group rather than project?
Our current workaround is to lower the parallel build from 6 to 1, but this apply to EVERY group, so there's only 1 queue to build over 500 projects. This slow down compilation considerably.

Thanks for any advice or suggestion.

--

Philippe Busque
1111, rue St-Charles Ouest,
Tour Est, bureau 255
Longueuil (Québec) Canada J4K 5G4
Tél. : 450-449-0102 ext. 3017
Télec. : 450-449-8725

Ce message et les fichiers d’accompagnement transmis avec celui-ci s’adressent expressément au(x) destinataire(s) et peuvent contenir des renseignements confidentiels et privilégiés. Si vous recevez ce message par erreur, veuillez en aviser immédiatement l’expéditeur par courrier électronique. Veuillez également ne pas en prendre connaissance et en supprimer toutes les copies immédiatement. Technologies Interactives Mediagrif Inc. et ses filiales n’acceptent aucune responsabilité à l’égard des opinions exprimées dans le message ou des conséquences de tout virus informatique qui pourrait être transmis avec ce message. Ce message fait également l’objet d’un copyright. Il est interdit d’en reproduire, adapter ou transmettre quelque partie que ce soit sans le consentement écrit du détenteur du copyright.

This email and any files transmitted with it are solely intended for the use of the addressee(s) and may contain information that is confidential and privileged. If you receive this email in error, please advise us by return email immediately. Please also disregard the contents of the email, delete it and destroy any copies immediately. Mediagrif Interactive Technologies Inc. and its subsidiaries do not accept liability for the views expressed in the email or for the consequences of any computer viruses that may be transmitted with this email. This email is also subject to copyright. No part of it should be reproduced, adapted or transmitted without the written consent of the copyright owner.

Re: Build dependencies issue with 1.4.1

[Brett Porter <br...@porterclan.net>]

Sorry for the delayed response, I thought I had seen a reply to this already.

The likely changes I can see would be:
http://svn.apache.org/viewvc/continuum/trunk/continuum-core/src/main/java/org/apache/continuum/buildmanager/ParallelBuildsManager.java?r1=1140475&r2=1153358&diff_format=h
http://svn.apache.org/viewvc/continuum/trunk/continuum-core/src/main/java/org/apache/continuum/buildmanager/ParallelBuildsManager.java?r1=798364&r2=946548&diff_format=h

I don't think this was intentional though - could you file an issue?

Continuum 1.4 put quite a bit more emphasis on distributed build. Have you tried using that? Even if it runs on the same server, you might like to try setting up some build agents - the grouping they afford gives much more capability than the parallel build option does.

Regards,
Brett

On 10/04/2013, at 3:07 AM, Philippe Busque <pb...@mediagrif.com> wrote:

> Hello,
> In January, we upgraded Continuum from version 1.3.8 to 1.4.1.  This helped solve one major issue we had with Continuum: speed due to permission pollution.
> 
> However, we've been getting more and more report from our developer than their change weren't in the daily build.  After investigation, we noticed that the build queue has changed since 1.3.8.
> 
> We have many project group. Each group contains a series of project that are dependant to each other.
> In 1.3.8,  when a scheduled build was triggered, Continuum would process each group, render the dependency and enqueue the builds in a unique queue per group.
> So group A would go in queue 1, group B in queue 2, group C in queue 1 and so on. We had 6 build queues and everything was built in order.
> 
> Since 1.4.1, we noticed the project were build out of order, more likely due to the fact that various project in groups were build into different queue. So project  A that depend on B who depend on C could be compiled with an older copy of C because it got queued after B and B compiled much faster than C.
> 
> Is there a way to restore the old way for queueing behaviour? Or enable parallel build based on group rather than project?
> Our current workaround is to lower the parallel build from 6 to 1, but this apply to EVERY group, so there's only 1 queue to build over 500 projects. This slow down compilation considerably.
> 
> Thanks for any advice or suggestion.
> 
> --
> 
> Philippe Busque
> 1111, rue St-Charles Ouest,
> Tour Est, bureau 255
> Longueuil (Québec) Canada J4K 5G4
> Tél. : 450-449-0102 ext. 3017
> Télec. : 450-449-8725
> 
> Ce message et les fichiers d’accompagnement transmis avec celui-ci s’adressent expressément au(x) destinataire(s) et peuvent contenir des renseignements confidentiels et privilégiés. Si vous recevez ce message par erreur, veuillez en aviser immédiatement l’expéditeur par courrier électronique. Veuillez également ne pas en prendre connaissance et en supprimer toutes les copies immédiatement. Technologies Interactives Mediagrif Inc. et ses filiales n’acceptent aucune responsabilité à l’égard des opinions exprimées dans le message ou des conséquences de tout virus informatique qui pourrait être transmis avec ce message. Ce message fait également l’objet d’un copyright. Il est interdit d’en reproduire, adapter ou transmettre quelque partie que ce soit sans le consentement écrit du détenteur du copyright.
> 
> This email and any files transmitted with it are solely intended for the use of the addressee(s) and may contain information that is confidential and privileged. If you receive this email in error, please advise us by return email immediately. Please also disregard the contents of the email, delete it and destroy any copies immediately. Mediagrif Interactive Technologies Inc. and its subsidiaries do not accept liability for the views expressed in the email or for the consequences of any computer viruses that may be transmitted with this email. This email is also subject to copyright. No part of it should be reproduced, adapted or transmitted without the written consent of the copyright owner.

--
Brett Porter
brett@apache.org
http://brettporter.wordpress.com/
http://au.linkedin.com/in/brettporter
http://twitter.com/brettporter