You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Pasquale Congiusti <pa...@gmail.com> on 2025/10/23 11:38:40 UTC
[HEADS UP] Some CI build/test stats time
Hi folks,
in the last days I'm analysing how to possibly improve our build process. I
thought it could be interesting to know which are the components that are
more costly to build. Here the stats:
[INFO] Camel :: Test Infra :: Cli (Camel JBang) ........... SUCCESS [34:58
min]
[INFO] Camel :: CXF :: SOAP :: Spring ..................... SUCCESS [17:17
min]
[INFO] Camel :: Management ................................ SUCCESS [12:14
min]
[INFO] Camel :: Spring RabbitMQ ........................... SUCCESS [11:15
min]
[INFO] Camel :: JMS ....................................... SUCCESS [11:09
min]
[INFO] Camel :: Kafka ..................................... SUCCESS [09:28
min]
[INFO] Camel :: Core ...................................... SUCCESS [09:14
min]
[INFO] Camel :: Zookeeper ................................. SUCCESS [09:10
min]
[INFO] Camel :: JBang :: Core ............................. SUCCESS [09:10
min]
[INFO] Camel :: Cassandra CQL ............................. SUCCESS [08:14
min]
[INFO] Camel :: AS2 :: Component .......................... SUCCESS [06:07
min]
[INFO] Camel :: FTP ....................................... SUCCESS [05:11
min]
[INFO] Camel :: Simple JMS ................................ SUCCESS [04:54
min]
[INFO] Camel :: SMB ....................................... SUCCESS [04:25
min]
[INFO] Camel :: Atmosphere WebSocket Servlet .............. SUCCESS [04:05
min]
[INFO] Camel :: Kubernetes ................................ SUCCESS [03:45
min]
[INFO] Camel :: Spring XML ................................ SUCCESS [03:40
min]
[INFO] Camel :: CXF :: REST :: Spring ..................... SUCCESS [03:34
min]
[INFO] Camel :: PQC ....................................... SUCCESS [03:18
min]
[INFO] Camel :: Avro RPC .................................. SUCCESS [02:45
min]
[INFO] Camel :: Endpoint DSL .............................. SUCCESS [02:27
min]
[INFO] Camel :: AS2 :: API ................................ SUCCESS [02:14
min]
[INFO] Camel :: Master .................................... SUCCESS [02:13
min]
[INFO] Camel :: Solr ...................................... SUCCESS [02:12
min]
[INFO] Camel :: Netty HTTP ................................ SUCCESS [02:12
min]
[INFO] Camel :: Integration Tests ......................... SUCCESS [02:05
min]
These are all the components which build and test takes longer than 2
minutes. I think that the ones on the top may require some review to
understand how reduce the time to build. Anybody with some time to
dedicate, feel free to have a look at those components which you may know
best. Consider the total time to build last between 4 and 5 hours, so, only
tackling the first 10 of this list would reduce the time to build by 2
hours (up to -50% on the total).
Feel free to reach out to me if you have any idea how to improve or you
need any further info.
Cheers,
Pasquale.
Re: [HEADS UP] Some CI build/test stats time
Posted by Aurelien Pupier <ap...@ibm.com.INVALID>.
Hello,
There was some work in the past related to this topic which can give also some ideas.
For instance:
https://issues.apache.org/jira/browse/CAMEL-19735
https://issues.apache.org/jira/browse/CAMEL-20886
https://issues.apache.org/jira/browse/CAMEL-20884 (there is a profiling of last investigated state where there is still room for improvements)
Also one of the biggest gain when I did a roud in the past was to remove some no more used/outdated features: the kotlin DSL and testing an old no more maintained Kafka version.
Regards,
________________________________
From: Pasquale Congiusti <pa...@gmail.com>
Sent: Thursday, October 23, 2025 1:38 PM
To: dev <de...@camel.apache.org>
Subject: [EXTERNAL] [HEADS UP] Some CI build/test stats time
Hi folks,
in the last days I'm analysing how to possibly improve our build process. I
thought it could be interesting to know which are the components that are
more costly to build. Here the stats:
[INFO] Camel :: Test Infra :: Cli (Camel JBang) ........... SUCCESS [34:58
min]
[INFO] Camel :: CXF :: SOAP :: Spring ..................... SUCCESS [17:17
min]
[INFO] Camel :: Management ................................ SUCCESS [12:14
min]
[INFO] Camel :: Spring RabbitMQ ........................... SUCCESS [11:15
min]
[INFO] Camel :: JMS ....................................... SUCCESS [11:09
min]
[INFO] Camel :: Kafka ..................................... SUCCESS [09:28
min]
[INFO] Camel :: Core ...................................... SUCCESS [09:14
min]
[INFO] Camel :: Zookeeper ................................. SUCCESS [09:10
min]
[INFO] Camel :: JBang :: Core ............................. SUCCESS [09:10
min]
[INFO] Camel :: Cassandra CQL ............................. SUCCESS [08:14
min]
[INFO] Camel :: AS2 :: Component .......................... SUCCESS [06:07
min]
[INFO] Camel :: FTP ....................................... SUCCESS [05:11
min]
[INFO] Camel :: Simple JMS ................................ SUCCESS [04:54
min]
[INFO] Camel :: SMB ....................................... SUCCESS [04:25
min]
[INFO] Camel :: Atmosphere WebSocket Servlet .............. SUCCESS [04:05
min]
[INFO] Camel :: Kubernetes ................................ SUCCESS [03:45
min]
[INFO] Camel :: Spring XML ................................ SUCCESS [03:40
min]
[INFO] Camel :: CXF :: REST :: Spring ..................... SUCCESS [03:34
min]
[INFO] Camel :: PQC ....................................... SUCCESS [03:18
min]
[INFO] Camel :: Avro RPC .................................. SUCCESS [02:45
min]
[INFO] Camel :: Endpoint DSL .............................. SUCCESS [02:27
min]
[INFO] Camel :: AS2 :: API ................................ SUCCESS [02:14
min]
[INFO] Camel :: Master .................................... SUCCESS [02:13
min]
[INFO] Camel :: Solr ...................................... SUCCESS [02:12
min]
[INFO] Camel :: Netty HTTP ................................ SUCCESS [02:12
min]
[INFO] Camel :: Integration Tests ......................... SUCCESS [02:05
min]
These are all the components which build and test takes longer than 2
minutes. I think that the ones on the top may require some review to
understand how reduce the time to build. Anybody with some time to
dedicate, feel free to have a look at those components which you may know
best. Consider the total time to build last between 4 and 5 hours, so, only
tackling the first 10 of this list would reduce the time to build by 2
hours (up to -50% on the total).
Feel free to reach out to me if you have any idea how to improve or you
need any further info.
Cheers,
Pasquale.
Unless otherwise stated above:
Compagnie IBM France
Siège Social : 17, avenue de l'Europe, 92275 Bois-Colombes Cedex
RCS Nanterre 552 118 465
Forme Sociale : S.A.S.
Capital Social : 664 614 175,50 €
SIRET : 552 118 465 03644 - Code NAF 6203Z
Re: [HEADS UP] Some CI build/test stats time
Posted by Pasquale Congiusti <pa...@gmail.com>.
Hello,
those figures are coming from Apache Jenkins builds. Those executions are
made in parallel, but I am pretty sure there is a lot of room for
improvement in each of the slowest modules and have a better responsiveness
overall.
Pasquale.
On Thu, Oct 23, 2025 at 3:35 PM Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> This is a bit slow CI server.
>
> You can for some components make them test in parallel via (like in
> camel-sql)
> Just mind that when doing so then the JVM is reused. To avoid this you can
> make it use fork (like in camel-management).
> But this also then depends on the number of cores the CI server gives you.
> But locally on your own computer then this can run much faster.
>
> <properties>
> <camel.surefire.parallel>true</camel.surefire.parallel>
> </properties>
>
>
> On Thu, Oct 23, 2025 at 1:39 PM Pasquale Congiusti <
> pasquale.congiusti@gmail.com> wrote:
>
> > Hi folks,
> > in the last days I'm analysing how to possibly improve our build
> process. I
> > thought it could be interesting to know which are the components that are
> > more costly to build. Here the stats:
> >
> > [INFO] Camel :: Test Infra :: Cli (Camel JBang) ........... SUCCESS
> [34:58
> > min]
> > [INFO] Camel :: CXF :: SOAP :: Spring ..................... SUCCESS
> [17:17
> > min]
> > [INFO] Camel :: Management ................................ SUCCESS
> [12:14
> > min]
> > [INFO] Camel :: Spring RabbitMQ ........................... SUCCESS
> [11:15
> > min]
> > [INFO] Camel :: JMS ....................................... SUCCESS
> [11:09
> > min]
> > [INFO] Camel :: Kafka ..................................... SUCCESS
> [09:28
> > min]
> > [INFO] Camel :: Core ...................................... SUCCESS
> [09:14
> > min]
> > [INFO] Camel :: Zookeeper ................................. SUCCESS
> [09:10
> > min]
> > [INFO] Camel :: JBang :: Core ............................. SUCCESS
> [09:10
> > min]
> > [INFO] Camel :: Cassandra CQL ............................. SUCCESS
> [08:14
> > min]
> > [INFO] Camel :: AS2 :: Component .......................... SUCCESS
> [06:07
> > min]
> > [INFO] Camel :: FTP ....................................... SUCCESS
> [05:11
> > min]
> > [INFO] Camel :: Simple JMS ................................ SUCCESS
> [04:54
> > min]
> > [INFO] Camel :: SMB ....................................... SUCCESS
> [04:25
> > min]
> > [INFO] Camel :: Atmosphere WebSocket Servlet .............. SUCCESS
> [04:05
> > min]
> > [INFO] Camel :: Kubernetes ................................ SUCCESS
> [03:45
> > min]
> > [INFO] Camel :: Spring XML ................................ SUCCESS
> [03:40
> > min]
> > [INFO] Camel :: CXF :: REST :: Spring ..................... SUCCESS
> [03:34
> > min]
> > [INFO] Camel :: PQC ....................................... SUCCESS
> [03:18
> > min]
> > [INFO] Camel :: Avro RPC .................................. SUCCESS
> [02:45
> > min]
> > [INFO] Camel :: Endpoint DSL .............................. SUCCESS
> [02:27
> > min]
> > [INFO] Camel :: AS2 :: API ................................ SUCCESS
> [02:14
> > min]
> > [INFO] Camel :: Master .................................... SUCCESS
> [02:13
> > min]
> > [INFO] Camel :: Solr ...................................... SUCCESS
> [02:12
> > min]
> > [INFO] Camel :: Netty HTTP ................................ SUCCESS
> [02:12
> > min]
> > [INFO] Camel :: Integration Tests ......................... SUCCESS
> [02:05
> > min]
> >
> > These are all the components which build and test takes longer than 2
> > minutes. I think that the ones on the top may require some review to
> > understand how reduce the time to build. Anybody with some time to
> > dedicate, feel free to have a look at those components which you may know
> > best. Consider the total time to build last between 4 and 5 hours, so,
> only
> > tackling the first 10 of this list would reduce the time to build by 2
> > hours (up to -50% on the total).
> >
> > Feel free to reach out to me if you have any idea how to improve or you
> > need any further info.
> >
> > Cheers,
> > Pasquale.
> >
>
>
> --
> Claus Ibsen
>
Re: [HEADS UP] Some CI build/test stats time
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
This is a bit slow CI server.
You can for some components make them test in parallel via (like in
camel-sql)
Just mind that when doing so then the JVM is reused. To avoid this you can
make it use fork (like in camel-management).
But this also then depends on the number of cores the CI server gives you.
But locally on your own computer then this can run much faster.
<properties>
<camel.surefire.parallel>true</camel.surefire.parallel>
</properties>
On Thu, Oct 23, 2025 at 1:39 PM Pasquale Congiusti <
pasquale.congiusti@gmail.com> wrote:
> Hi folks,
> in the last days I'm analysing how to possibly improve our build process. I
> thought it could be interesting to know which are the components that are
> more costly to build. Here the stats:
>
> [INFO] Camel :: Test Infra :: Cli (Camel JBang) ........... SUCCESS [34:58
> min]
> [INFO] Camel :: CXF :: SOAP :: Spring ..................... SUCCESS [17:17
> min]
> [INFO] Camel :: Management ................................ SUCCESS [12:14
> min]
> [INFO] Camel :: Spring RabbitMQ ........................... SUCCESS [11:15
> min]
> [INFO] Camel :: JMS ....................................... SUCCESS [11:09
> min]
> [INFO] Camel :: Kafka ..................................... SUCCESS [09:28
> min]
> [INFO] Camel :: Core ...................................... SUCCESS [09:14
> min]
> [INFO] Camel :: Zookeeper ................................. SUCCESS [09:10
> min]
> [INFO] Camel :: JBang :: Core ............................. SUCCESS [09:10
> min]
> [INFO] Camel :: Cassandra CQL ............................. SUCCESS [08:14
> min]
> [INFO] Camel :: AS2 :: Component .......................... SUCCESS [06:07
> min]
> [INFO] Camel :: FTP ....................................... SUCCESS [05:11
> min]
> [INFO] Camel :: Simple JMS ................................ SUCCESS [04:54
> min]
> [INFO] Camel :: SMB ....................................... SUCCESS [04:25
> min]
> [INFO] Camel :: Atmosphere WebSocket Servlet .............. SUCCESS [04:05
> min]
> [INFO] Camel :: Kubernetes ................................ SUCCESS [03:45
> min]
> [INFO] Camel :: Spring XML ................................ SUCCESS [03:40
> min]
> [INFO] Camel :: CXF :: REST :: Spring ..................... SUCCESS [03:34
> min]
> [INFO] Camel :: PQC ....................................... SUCCESS [03:18
> min]
> [INFO] Camel :: Avro RPC .................................. SUCCESS [02:45
> min]
> [INFO] Camel :: Endpoint DSL .............................. SUCCESS [02:27
> min]
> [INFO] Camel :: AS2 :: API ................................ SUCCESS [02:14
> min]
> [INFO] Camel :: Master .................................... SUCCESS [02:13
> min]
> [INFO] Camel :: Solr ...................................... SUCCESS [02:12
> min]
> [INFO] Camel :: Netty HTTP ................................ SUCCESS [02:12
> min]
> [INFO] Camel :: Integration Tests ......................... SUCCESS [02:05
> min]
>
> These are all the components which build and test takes longer than 2
> minutes. I think that the ones on the top may require some review to
> understand how reduce the time to build. Anybody with some time to
> dedicate, feel free to have a look at those components which you may know
> best. Consider the total time to build last between 4 and 5 hours, so, only
> tackling the first 10 of this list would reduce the time to build by 2
> hours (up to -50% on the total).
>
> Feel free to reach out to me if you have any idea how to improve or you
> need any further info.
>
> Cheers,
> Pasquale.
>
--
Claus Ibsen