Re: JsonReader.readObject() fails with ArrayIndexOutOfBoundsException when a string is too long, but is full of backslashes?

[Kean Erickson <ke...@gmail.com>]

Sorry to bump this but it looks like this is a much more general problem
than I thought. Previously I was having an issue when reading a file with
too many backslashes, and the fix was to acquire my JsonReader from a
JsonReaderFactory created with a settings map that overrides
JsonParserFactoryImpl.MAX_STRING_LENGTH with a correct value so that the
file read wouldn't explode.

But now I'm noticing that it's possible for a simple POST or PUT to a
resource with a String value of the attached contents blows up with the
same "java.lang.ArrayIndexOutOfBoundsException: 8192" error message from
before, due to having just the right number of backslahes from tabs, only
it doesn't seem like I have any control in this case to override the
maximum length that is being used. Like I mentioned previously, I think the
bug is that in JsonParserFactoryImpl, a system level maxSize value is set
to default to 262144 if the max-string-length johnzon property isn't set as
a system property. But before this, CxfRSService sets this in a system
property as 8192. So the larger value 262144 never applies.

This unconfigurable string length limitation is a problem for any API we
create that will use Johnzon to receive a string, do y'all know of any kind
of workaround (like a system-level setting)?

Thanks,
-Kean

On Tue, Feb 26, 2019 at 5:45 PM Kean Erickson <ke...@gmail.com>
wrote:

> I swapped to 7.1.0 and am seeing the same thing. But I think I have a
> better idea of what is going on by default when this property isn't
> configured manually.
>
> In JsonParserFactoryImpl, a system level maxSize value is set and to
> default to 262144 if the max-string-length johnzon property isn't set as a
> system property. But before this, CxfRSService sets this in a system
> property as 8192. So the larger value 262144 never applies.
>
> On Tue, Feb 26, 2019 at 1:44 PM Mark Struberg <st...@yahoo.de.invalid>
> wrote:
>
>> We did tons of fixes in Johnzon. Can you please try out TomEE-7.0.5? This
>> version contains a much newer version of Johnzon.
>>
>> LieGrue,
>> strub
>>
>>
>> > Am 26.02.2019 um 20:54 schrieb Kean Erickson <ke...@gmail.com>:
>> >
>> > Sorry to reply to my own email but one more question on that--if I am to
>> > set "org.apache.johnzon.max-string-length", is there anywhere that the
>> > default value for this is documented? The error message claims the
>> default
>> > is 8192, but if I override it in system.properties with 4092 the warning
>> > goes away even though that would have resulted in the limit being
>> lowered
>> > if the "Too many characters" error message were correct.. So I'm not
>> really
>> > sure what's going on.
>> >
>> > On Mon, Feb 25, 2019 at 5:19 PM Kean Erickson <ke...@gmail.com>
>> > wrote:
>> >
>> >> I'm using TomEE webprofile 7.04 in an exploded WAR deployment. I need
>> to
>> >> use JsonReader's readObject() to read from a JsonReader created from a
>> >> string, but the JsonReader's readObject() call fails ineloquently when
>> the
>> >> object read is too long. I unfortunately haven't been able to find a
>> way to
>> >> make a unit test that reproduces this, it always passes even if the
>> string
>> >> length is over this limit.. I think being scoped to a request may have
>> >> something to do with it.
>> >>
>> >> After putting the attached bad.json in my user directory, I acquire the
>> >> attached file's contents as such:
>> >>
>> >> File bad = new File(System.getProperty("user.dir")+"/bad.json");
>> >> byte[] badbytes = Files.readAllBytes(bad.toPath());
>> >> String badStr =  new String(badbytes, StandardCharsets.UTF_8);
>> >> try(JsonReader jr = Json.createReader(new StringReader(badStr))) {
>> >>    JsonObject json = jr.readObject();
>> >> }
>> >>
>> >> From this I get the error:
>> >>
>> >> "java.lang.ArrayIndexOutOfBoundsException: 8192
>> >>    at
>> >>
>> org.apache.johnzon.core.JsonStreamParserImpl.appendToCopyBuffer(JsonStreamParserImpl.java:156)
>> >>    at
>> >>
>> org.apache.johnzon.core.JsonStreamParserImpl.readString(JsonStreamParserImpl.java:581)
>> >>    at
>> >>
>> org.apache.johnzon.core.JsonStreamParserImpl.handleQuote(JsonStreamParserImpl.java:684)
>> >>    at
>> >>
>> org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:429)
>> >>    at
>> >>
>> org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:389)
>> >>    at
>> >>
>> org.apache.johnzon.core.JsonReaderImpl.parseObject(JsonReaderImpl.java:181)
>> >>    at
>> >>
>> org.apache.johnzon.core.JsonReaderImpl.readValue(JsonReaderImpl.java:82)
>> >>    at
>> org.apache.johnzon.core.JsonReaderImpl.read(JsonReaderImpl.java:61)
>> >>    at
>> >>
>> org.apache.johnzon.core.JsonReaderImpl.readObject(JsonReaderImpl.java:150)"
>> >>
>> >>
>> >> ....but if I replace all the backslashes in that file with a standard
>> >> character like "a", I get the error message that is probably intended
>> for
>> >> the above case:
>> >>
>> >> Too many characters. Maximum string/number length of 8192 exceeded on
>> >> [lineNumber=1, columnNumber=10591, streamOffset=10590]. Maybe increase
>> >> org.apache.johnzon.max-string-length in jsonp factory properties or
>> system
>> >> properties.
>> >>
>> >>
>> >> I think there's a bug where this message doesn't appear if when there
>> are
>> >> a lot of escape characters (slashes) that bring the string length over
>> the
>> >> limit. Any reason this proper warning wouldn't appear when triggered by
>> >> escape characters? This is occurring both on Windows and Unix.
>> >>
>> >> Thanks,
>> >> -Kean
>> >>
>>
>>