You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@continuum.apache.org by "Philippe Busque (JIRA)" <ji...@codehaus.org> on 2010/04/29 19:41:12 UTC
[jira] Created: (CONTINUUM-2514) Prepare Build Queue slow down
parallel built start
Prepare Build Queue slow down parallel built start
--------------------------------------------------
Key: CONTINUUM-2514
URL: http://jira.codehaus.org/browse/CONTINUUM-2514
Project: Continuum
Issue Type: Bug
Components: Parallel Builds
Affects Versions: 1.3.6
Reporter: Philippe Busque
When a scheduler or a build group trigger a series of compilation, every projects are enqueued in the Prepare Build Queue. This queue doesn't consider the project group of a project when iterating through it.
As a result, newly added project, but independent, can wait a long time before being launched in a different built queue, as it has to wait for every entries from the previous groups to be processed.
To speed up the launch of parallel builts, the Prepare Build Queue should alternate between Project Group when more than 1 group is present.
Example:
Set up 4 parallel built queues.
Put 100 projects in Group A.
Put 2 projects in Group B.
Launch the built of Group A. All jobs are inserted in the Prepare Build Queue.
Launch the built of Group B. The 2 jobs are inserted in the Prepare Build Queue.
Since the projects of group B are independent to Group A, they should be added to the Built Queue immediately. However, they will only be added once the 100 projects from Group A has been analyzed.
In our case, we have 6 queues and 40 groups. The average busy queue for the first half hour of compile is 4 queues as the prepare queue doesn't push new projects fast enough to fill the queues.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CONTINUUM-2514) Prepare Build Queue slow down
parallel built start
Posted by "Maria Catherine Tan (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/CONTINUUM-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maria Catherine Tan updated CONTINUUM-2514:
-------------------------------------------
Fix Version/s: (was: 1.4.1 (Beta))
1.4.2 (Beta)
> Prepare Build Queue slow down parallel built start
> --------------------------------------------------
>
> Key: CONTINUUM-2514
> URL: https://jira.codehaus.org/browse/CONTINUUM-2514
> Project: Continuum
> Issue Type: Bug
> Components: Parallel Builds
> Affects Versions: 1.3.6
> Reporter: Philippe Busque
> Fix For: 1.4.2 (Beta)
>
>
> When a scheduler or a build group trigger a series of compilation, every projects are enqueued in the Prepare Build Queue. This queue doesn't consider the project group of a project when iterating through it.
> As a result, newly added project, but independent, can wait a long time before being launched in a different built queue, as it has to wait for every entries from the previous groups to be processed.
> To speed up the launch of parallel builts, the Prepare Build Queue should alternate between Project Group when more than 1 group is present.
> Example:
> Set up 4 parallel built queues.
> Put 100 projects in Group A.
> Put 2 projects in Group B.
> Launch the built of Group A. All jobs are inserted in the Prepare Build Queue.
> Launch the built of Group B. The 2 jobs are inserted in the Prepare Build Queue.
> Since the projects of group B are independent to Group A, they should be added to the Built Queue immediately. However, they will only be added once the 100 projects from Group A has been analyzed.
> In our case, we have 6 queues and 40 groups. The average busy queue for the first half hour of compile is 4 queues as the prepare queue doesn't push new projects fast enough to fill the queues.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CONTINUUM-2514) Prepare Build Queue slow down
parallel built start
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/CONTINUUM-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brett Porter updated CONTINUUM-2514:
------------------------------------
Fix Version/s: 1.4.1 (Beta)
> Prepare Build Queue slow down parallel built start
> --------------------------------------------------
>
> Key: CONTINUUM-2514
> URL: http://jira.codehaus.org/browse/CONTINUUM-2514
> Project: Continuum
> Issue Type: Bug
> Components: Parallel Builds
> Affects Versions: 1.3.6
> Reporter: Philippe Busque
> Fix For: 1.4.1 (Beta)
>
>
> When a scheduler or a build group trigger a series of compilation, every projects are enqueued in the Prepare Build Queue. This queue doesn't consider the project group of a project when iterating through it.
> As a result, newly added project, but independent, can wait a long time before being launched in a different built queue, as it has to wait for every entries from the previous groups to be processed.
> To speed up the launch of parallel builts, the Prepare Build Queue should alternate between Project Group when more than 1 group is present.
> Example:
> Set up 4 parallel built queues.
> Put 100 projects in Group A.
> Put 2 projects in Group B.
> Launch the built of Group A. All jobs are inserted in the Prepare Build Queue.
> Launch the built of Group B. The 2 jobs are inserted in the Prepare Build Queue.
> Since the projects of group B are independent to Group A, they should be added to the Built Queue immediately. However, they will only be added once the 100 projects from Group A has been analyzed.
> In our case, we have 6 queues and 40 groups. The average busy queue for the first half hour of compile is 4 queues as the prepare queue doesn't push new projects fast enough to fill the queues.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira