Re: buildbot failure in on tomcat-9-trunk

[Mark Thomas <ma...@apache.org>]

On 22/04/2020 22:53, buildbot@apache.org wrote:
> The Buildbot has detected a new failure on builder tomcat-9-trunk while building tomcat. Full details are available at:
>     https://ci.apache.org/builders/tomcat-9-trunk/builds/193

This is bizarre.

There are two failures but the really interesting one is this one:

https://ci.apache.org/projects/tomcat/tomcat9/logs/193/TEST-org.apache.coyote.http2.TestAsyncTimeout.NIO.txt

The unprintable characters in the output are all 0x00.

The buffer is 8192 bytes in size.

This write starts at position 8032 rather than 0.

The stack trace indicates buffer overflow. But...

Line 819 of OutputBuffer explicitly checks for the remaining space and
so should avoid this issue.

There is a single test in this class and Ant is configured to use a new
JVM for evert test class so this is the first and only request to be
processed by this JVM. Therefore:
- a failure in clean-up after re-use cannot be the root cause as there
  are no prior requests;
- I don't see a code path where the position can be anything other than
  zero when the writes start.

If arrayOffset was not zero that might explain the behaviour but
CharBuffer.allocate(size) is documented to return a CharBuffer with
arrayOffset of zero.

I'm running out of explanations that don't involve rare JVM bugs, faulty
hardware, etc.

Anyone got any ideas?

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org

Re: buildbot failure in on tomcat-9-trunk

[Rémy Maucherat <re...@apache.org>]

On Thu, Apr 23, 2020 at 12:02 PM Mark Thomas <ma...@apache.org> wrote:

> On 22/04/2020 22:53, buildbot@apache.org wrote:
> > The Buildbot has detected a new failure on builder tomcat-9-trunk while
> building tomcat. Full details are available at:
> >     https://ci.apache.org/builders/tomcat-9-trunk/builds/193
>
> This is bizarre.
>
> There are two failures but the really interesting one is this one:
>
>
> https://ci.apache.org/projects/tomcat/tomcat9/logs/193/TEST-org.apache.coyote.http2.TestAsyncTimeout.NIO.txt
>
> The unprintable characters in the output are all 0x00.
>
> The buffer is 8192 bytes in size.
>
> This write starts at position 8032 rather than 0.
>
> The stack trace indicates buffer overflow. But...
>
> Line 819 of OutputBuffer explicitly checks for the remaining space and
> so should avoid this issue.
>
> There is a single test in this class and Ant is configured to use a new
> JVM for evert test class so this is the first and only request to be
> processed by this JVM. Therefore:
> - a failure in clean-up after re-use cannot be the root cause as there
>   are no prior requests;
> - I don't see a code path where the position can be anything other than
>   zero when the writes start.
>
> If arrayOffset was not zero that might explain the behaviour but
> CharBuffer.allocate(size) is documented to return a CharBuffer with
> arrayOffset of zero.
>
> I'm running out of explanations that don't involve rare JVM bugs, faulty
> hardware, etc.
>
> Anyone got any ideas?
>

I think it's good to check failures (I do it often), but it's hard to come
up with something here. Usually problems don't occur in the Catalina
connector, which has been stable for a long while. Odd.
So the only possible explanation besides "rare JVM bugs, faulty hardware"
is a concurrent write, which is not protected against [that is normal,
since it usually won't work even if you take out IO corruption]. But I
don't really see a way to get into a concurrent write here.

Rémy


>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>