You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Ripu Daman <ri...@gmail.com> on 2020/06/09 12:03:39 UTC
Tomcat 9.0.0 multiple thread issue
Hello everyone,
I work in an IT firm as a software engineer. I don't know whether it's a
right platform to ask questions based on personal projects but i believe i
will get some help from here.
We are facing an issue regarding multiple threads being created for a
servlet execution running in tomcat 9.0.0
Servlet's job is to publish customer leads to salesforce via soap protocol.
The logic has been designed in such a way that in a single thread it can
publish multiple leads to salesforce one by one. However we recently
migrated our java application to kubernetes adopting containerized
approach.
Post this we are facing an issue in which, when the process starts
publishing leads to salesforce one by one, after a few uploads a new thread
gets created and it starts the whole process from beginning and then after
a few seconds another thread kicks off repeating the whole process. It
keeps going on until session timeout exceeds and we get a 504 error. But
the upload process keeps going on in the background and stops after 2-3
threads.
Before moving to kubernetes, the process was working fine. In order to
serve multiple requests at the same time, servlet is not designed to be
thread safe.
But now we are facing this issue and unable to identify the root cause.
Here's a sample of threads from the logs :
[http-nio-8080-exec-9] INFO
[http-nio-8080-exec-9] INFO
[http-nio-8080-exec-9] INFO
[http-nio-8080-exec-7] INFO
[http-nio-8080-exec-7] INFO
[http-nio-8080-exec-9] INFO
[http-nio-8080-exec-9] INFO
[http-nio-8080-exec-7] INFO
[http-nio-8080-exec-7] INFO
[http-nio-8080-exec-7] INFO
[http-nio-8080-exec-3] INFO
[http-nio-8080-exec-7] INFO
[http-nio-8080-exec-3] INFO
The configurations for tomcat remains same as default. Session timeout is
set as 30 min (default).
For more info please let me know.
Hope will get some help from here.
Thanks!
regards,
Ripu daman
Re: Tomcat 9.0.0 multiple thread issue
Posted by Ripu Daman <ri...@gmail.com>.
Sorry for late reply. We have fixed the issue. Kubernetes ingress file was
causing multiple requests to be sent over and over again. Proxy timeout
along with read and send timeouts were not present in that file. We have
added them and now the process is working fine allowing it to complete with
only one single thread.
Thank you all for your support.
Regards,
Ripu daman
On Tue 9 Jun, 2020, 10:20 PM Christopher Schultz, <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Ripu,
>
> On 6/9/20 08:03, Ripu Daman wrote:
> > Hello everyone,
> >
> > I work in an IT firm as a software engineer. I don't know whether
> > it's a right platform to ask questions based on personal projects
> > but i believe i will get some help from here.
> >
> > We are facing an issue regarding multiple threads being created for
> > a servlet execution running in tomcat 9.0.0
> >
> > Servlet's job is to publish customer leads to salesforce via soap
> > protocol. The logic has been designed in such a way that in a
> > single thread it can publish multiple leads to salesforce one by
> > one. However we recently migrated our java application to
> > kubernetes adopting containerized approach. Post this we are facing
> > an issue in which, when the process starts publishing leads to
> > salesforce one by one, after a few uploads a new thread gets
> > created and it starts the whole process from beginning and then
> > after a few seconds another thread kicks off repeating the whole
> > process. It keeps going on until session timeout exceeds and we get
> > a 504 error. But the upload process keeps going on in the
> > background and stops after 2-3 threads.
>
> Where is your queue stored?
>
> > Before moving to kubernetes, the process was working fine. In order
> > to serve multiple requests at the same time, servlet is not
> > designed to be thread safe.
>
> It's NOT designed to be thread-safe? I hope you meant it IS designed
> to be thread-safe.
>
> > But now we are facing this issue and unable to identify the root
> > cause.
> >
> > Here's a sample of threads from the logs : [http-nio-8080-exec-9]
> > INFO [http-nio-8080-exec-9] INFO [http-nio-8080-exec-9] INFO
> > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-7] INFO
> > [http-nio-8080-exec-9] INFO [http-nio-8080-exec-9] INFO
> >
> > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-7] INFO
> > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-3] INFO
> > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-3] INFO
>
> These are request-processing threads, so something is making HTTP
> requests to Tomcat to do that work.
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl7fvbEACgkQHPApP6U8
> pFi9Uw/+O9Be+Bp7n8co6Rz7gkvtXZbKT5KlxpWtT3gVNluN4Lt824Hgb5C7wmMd
> AjZfvFO0yn1rIJsY+G0r/25lsdIVLZZW3ehM28eRgc+Up3sI66w8zvnOGhly7E6f
> CKvSKkINLrS9+BQ0HrLkqHxn+U4y2Ts8yfRZOelC8dGtFfj+U3cSsc75LGMUiREM
> SlYabv4dDzemfBqG/08zXfQ+laQStg9Cf+rY//ZZGnLWR74obJJB/rlhrNMAzND0
> qR5Z0LebYXGQEt6nElYtDtAosJlnlpX30fLX6aVwwIXfgPqeIgBF9PskIfwx1NU5
> sF3nIHO4wE+KsMqM22fqMOU6keBR/Ac8ktMEws/d6PEL1oXUhcsXMTulh6RrxHWA
> ImTlksYyIp54flIw3cwiooVW1VrgYQisIIucsWPzLSw7QsE9CyZADqYYm6N7Eo60
> VpdHAOLTX4zfJoMlf7NWuMdK8dFI8wkLUmCURxYkWysWx+MJ55BDgDy9kTw3QZoa
> lBZs9bPirGU8r7qmjbrj5L/Q0lpeqQSSj9ixBzAJKGBAYzYJHKWPpOcI2NW1hn6F
> vUi01xB67zXboH1EcLVHqMc9srA5UjaDKzwnCcwde1kBaxRbwHFIOq1s+BflXdEc
> 69gaaPqifMnp1pLaJoonPDyZ3QszSAm5SiUi6lKmofF0UBqTzYs=
> =SxID
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Tomcat 9.0.0 multiple thread issue
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Ripu,
On 6/9/20 08:03, Ripu Daman wrote:
> Hello everyone,
>
> I work in an IT firm as a software engineer. I don't know whether
> it's a right platform to ask questions based on personal projects
> but i believe i will get some help from here.
>
> We are facing an issue regarding multiple threads being created for
> a servlet execution running in tomcat 9.0.0
>
> Servlet's job is to publish customer leads to salesforce via soap
> protocol. The logic has been designed in such a way that in a
> single thread it can publish multiple leads to salesforce one by
> one. However we recently migrated our java application to
> kubernetes adopting containerized approach. Post this we are facing
> an issue in which, when the process starts publishing leads to
> salesforce one by one, after a few uploads a new thread gets
> created and it starts the whole process from beginning and then
> after a few seconds another thread kicks off repeating the whole
> process. It keeps going on until session timeout exceeds and we get
> a 504 error. But the upload process keeps going on in the
> background and stops after 2-3 threads.
Where is your queue stored?
> Before moving to kubernetes, the process was working fine. In order
> to serve multiple requests at the same time, servlet is not
> designed to be thread safe.
It's NOT designed to be thread-safe? I hope you meant it IS designed
to be thread-safe.
> But now we are facing this issue and unable to identify the root
> cause.
>
> Here's a sample of threads from the logs : [http-nio-8080-exec-9]
> INFO [http-nio-8080-exec-9] INFO [http-nio-8080-exec-9] INFO
> [http-nio-8080-exec-7] INFO [http-nio-8080-exec-7] INFO
> [http-nio-8080-exec-9] INFO [http-nio-8080-exec-9] INFO
>
> [http-nio-8080-exec-7] INFO [http-nio-8080-exec-7] INFO
> [http-nio-8080-exec-7] INFO [http-nio-8080-exec-3] INFO
> [http-nio-8080-exec-7] INFO [http-nio-8080-exec-3] INFO
These are request-processing threads, so something is making HTTP
requests to Tomcat to do that work.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl7fvbEACgkQHPApP6U8
pFi9Uw/+O9Be+Bp7n8co6Rz7gkvtXZbKT5KlxpWtT3gVNluN4Lt824Hgb5C7wmMd
AjZfvFO0yn1rIJsY+G0r/25lsdIVLZZW3ehM28eRgc+Up3sI66w8zvnOGhly7E6f
CKvSKkINLrS9+BQ0HrLkqHxn+U4y2Ts8yfRZOelC8dGtFfj+U3cSsc75LGMUiREM
SlYabv4dDzemfBqG/08zXfQ+laQStg9Cf+rY//ZZGnLWR74obJJB/rlhrNMAzND0
qR5Z0LebYXGQEt6nElYtDtAosJlnlpX30fLX6aVwwIXfgPqeIgBF9PskIfwx1NU5
sF3nIHO4wE+KsMqM22fqMOU6keBR/Ac8ktMEws/d6PEL1oXUhcsXMTulh6RrxHWA
ImTlksYyIp54flIw3cwiooVW1VrgYQisIIucsWPzLSw7QsE9CyZADqYYm6N7Eo60
VpdHAOLTX4zfJoMlf7NWuMdK8dFI8wkLUmCURxYkWysWx+MJ55BDgDy9kTw3QZoa
lBZs9bPirGU8r7qmjbrj5L/Q0lpeqQSSj9ixBzAJKGBAYzYJHKWPpOcI2NW1hn6F
vUi01xB67zXboH1EcLVHqMc9srA5UjaDKzwnCcwde1kBaxRbwHFIOq1s+BflXdEc
69gaaPqifMnp1pLaJoonPDyZ3QszSAm5SiUi6lKmofF0UBqTzYs=
=SxID
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 9.0.0 multiple thread issue
Posted by Paul Carter-Brown <pa...@jini.guru>.
I assume a servlet request is kicking off this process? If this is correct
then are you 100% sure that something is not calling the servlet more than
once? Perhaps a retry after a timeout or something?
Turn your access log on and look at what requests are coming in to the
server
Paul
On Tue, Jun 9, 2020 at 2:04 PM Ripu Daman <ri...@gmail.com> wrote:
> Hello everyone,
>
> I work in an IT firm as a software engineer. I don't know whether it's a
> right platform to ask questions based on personal projects but i believe i
> will get some help from here.
>
> We are facing an issue regarding multiple threads being created for a
> servlet execution running in tomcat 9.0.0
>
> Servlet's job is to publish customer leads to salesforce via soap protocol.
> The logic has been designed in such a way that in a single thread it can
> publish multiple leads to salesforce one by one. However we recently
> migrated our java application to kubernetes adopting containerized
> approach.
> Post this we are facing an issue in which, when the process starts
> publishing leads to salesforce one by one, after a few uploads a new thread
> gets created and it starts the whole process from beginning and then after
> a few seconds another thread kicks off repeating the whole process. It
> keeps going on until session timeout exceeds and we get a 504 error. But
> the upload process keeps going on in the background and stops after 2-3
> threads.
>
> Before moving to kubernetes, the process was working fine. In order to
> serve multiple requests at the same time, servlet is not designed to be
> thread safe.
>
> But now we are facing this issue and unable to identify the root cause.
>
> Here's a sample of threads from the logs :
> [http-nio-8080-exec-9] INFO
> [http-nio-8080-exec-9] INFO
> [http-nio-8080-exec-9] INFO
> [http-nio-8080-exec-7] INFO
> [http-nio-8080-exec-7] INFO
> [http-nio-8080-exec-9] INFO
> [http-nio-8080-exec-9] INFO
>
> [http-nio-8080-exec-7] INFO
> [http-nio-8080-exec-7] INFO
> [http-nio-8080-exec-7] INFO
> [http-nio-8080-exec-3] INFO
> [http-nio-8080-exec-7] INFO
> [http-nio-8080-exec-3] INFO
>
> The configurations for tomcat remains same as default. Session timeout is
> set as 30 min (default).
>
> For more info please let me know.
>
> Hope will get some help from here.
>
> Thanks!
>
> regards,
> Ripu daman
>