You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomee.apache.org by Kean Erickson <ke...@gmail.com> on 2019/02/26 01:19:34 UTC
JsonReader.readObject() fails with ArrayIndexOutOfBoundsException
when a string is too long, but is full of backslashes?
I'm using TomEE webprofile 7.04 in an exploded WAR deployment. I need to
use JsonReader's readObject() to read from a JsonReader created from a
string, but the JsonReader's readObject() call fails ineloquently when the
object read is too long. I unfortunately haven't been able to find a way to
make a unit test that reproduces this, it always passes even if the string
length is over this limit.. I think being scoped to a request may have
something to do with it.
After putting the attached bad.json in my user directory, I acquire the
attached file's contents as such:
File bad = new File(System.getProperty("user.dir")+"/bad.json");
byte[] badbytes = Files.readAllBytes(bad.toPath());
String badStr = new String(badbytes, StandardCharsets.UTF_8);
try(JsonReader jr = Json.createReader(new StringReader(badStr))) {
JsonObject json = jr.readObject();
}
From this I get the error:
"java.lang.ArrayIndexOutOfBoundsException: 8192
at
org.apache.johnzon.core.JsonStreamParserImpl.appendToCopyBuffer(JsonStreamParserImpl.java:156)
at
org.apache.johnzon.core.JsonStreamParserImpl.readString(JsonStreamParserImpl.java:581)
at
org.apache.johnzon.core.JsonStreamParserImpl.handleQuote(JsonStreamParserImpl.java:684)
at
org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:429)
at
org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:389)
at
org.apache.johnzon.core.JsonReaderImpl.parseObject(JsonReaderImpl.java:181)
at
org.apache.johnzon.core.JsonReaderImpl.readValue(JsonReaderImpl.java:82)
at org.apache.johnzon.core.JsonReaderImpl.read(JsonReaderImpl.java:61)
at
org.apache.johnzon.core.JsonReaderImpl.readObject(JsonReaderImpl.java:150)"
....but if I replace all the backslashes in that file with a standard
character like "a", I get the error message that is probably intended for
the above case:
Too many characters. Maximum string/number length of 8192 exceeded on
[lineNumber=1, columnNumber=10591, streamOffset=10590]. Maybe increase
org.apache.johnzon.max-string-length in jsonp factory properties or system
properties.
I think there's a bug where this message doesn't appear if when there are a
lot of escape characters (slashes) that bring the string length over the
limit. Any reason this proper warning wouldn't appear when triggered by
escape characters? This is occurring both on Windows and Unix.
Thanks,
-Kean
Re: JsonReader.readObject() fails with ArrayIndexOutOfBoundsException
when a string is too long, but is full of backslashes?
Posted by Kean Erickson <ke...@gmail.com>.
Sorry to bump this but it looks like this is a much more general problem
than I thought. Previously I was having an issue when reading a file with
too many backslashes, and the fix was to acquire my JsonReader from a
JsonReaderFactory created with a settings map that overrides
JsonParserFactoryImpl.MAX_STRING_LENGTH with a correct value so that the
file read wouldn't explode.
But now I'm noticing that it's possible for a simple POST or PUT to a
resource with a String value of the attached contents blows up with the
same "java.lang.ArrayIndexOutOfBoundsException: 8192" error message from
before, due to having just the right number of backslahes from tabs, only
it doesn't seem like I have any control in this case to override the
maximum length that is being used. Like I mentioned previously, I think the
bug is that in JsonParserFactoryImpl, a system level maxSize value is set
to default to 262144 if the max-string-length johnzon property isn't set as
a system property. But before this, CxfRSService sets this in a system
property as 8192. So the larger value 262144 never applies.
This unconfigurable string length limitation is a problem for any API we
create that will use Johnzon to receive a string, do y'all know of any kind
of workaround (like a system-level setting)?
Thanks,
-Kean
On Tue, Feb 26, 2019 at 5:45 PM Kean Erickson <ke...@gmail.com>
wrote:
> I swapped to 7.1.0 and am seeing the same thing. But I think I have a
> better idea of what is going on by default when this property isn't
> configured manually.
>
> In JsonParserFactoryImpl, a system level maxSize value is set and to
> default to 262144 if the max-string-length johnzon property isn't set as a
> system property. But before this, CxfRSService sets this in a system
> property as 8192. So the larger value 262144 never applies.
>
> On Tue, Feb 26, 2019 at 1:44 PM Mark Struberg <st...@yahoo.de.invalid>
> wrote:
>
>> We did tons of fixes in Johnzon. Can you please try out TomEE-7.0.5? This
>> version contains a much newer version of Johnzon.
>>
>> LieGrue,
>> strub
>>
>>
>> > Am 26.02.2019 um 20:54 schrieb Kean Erickson <ke...@gmail.com>:
>> >
>> > Sorry to reply to my own email but one more question on that--if I am to
>> > set "org.apache.johnzon.max-string-length", is there anywhere that the
>> > default value for this is documented? The error message claims the
>> default
>> > is 8192, but if I override it in system.properties with 4092 the warning
>> > goes away even though that would have resulted in the limit being
>> lowered
>> > if the "Too many characters" error message were correct.. So I'm not
>> really
>> > sure what's going on.
>> >
>> > On Mon, Feb 25, 2019 at 5:19 PM Kean Erickson <ke...@gmail.com>
>> > wrote:
>> >
>> >> I'm using TomEE webprofile 7.04 in an exploded WAR deployment. I need
>> to
>> >> use JsonReader's readObject() to read from a JsonReader created from a
>> >> string, but the JsonReader's readObject() call fails ineloquently when
>> the
>> >> object read is too long. I unfortunately haven't been able to find a
>> way to
>> >> make a unit test that reproduces this, it always passes even if the
>> string
>> >> length is over this limit.. I think being scoped to a request may have
>> >> something to do with it.
>> >>
>> >> After putting the attached bad.json in my user directory, I acquire the
>> >> attached file's contents as such:
>> >>
>> >> File bad = new File(System.getProperty("user.dir")+"/bad.json");
>> >> byte[] badbytes = Files.readAllBytes(bad.toPath());
>> >> String badStr = new String(badbytes, StandardCharsets.UTF_8);
>> >> try(JsonReader jr = Json.createReader(new StringReader(badStr))) {
>> >> JsonObject json = jr.readObject();
>> >> }
>> >>
>> >> From this I get the error:
>> >>
>> >> "java.lang.ArrayIndexOutOfBoundsException: 8192
>> >> at
>> >>
>> org.apache.johnzon.core.JsonStreamParserImpl.appendToCopyBuffer(JsonStreamParserImpl.java:156)
>> >> at
>> >>
>> org.apache.johnzon.core.JsonStreamParserImpl.readString(JsonStreamParserImpl.java:581)
>> >> at
>> >>
>> org.apache.johnzon.core.JsonStreamParserImpl.handleQuote(JsonStreamParserImpl.java:684)
>> >> at
>> >>
>> org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:429)
>> >> at
>> >>
>> org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:389)
>> >> at
>> >>
>> org.apache.johnzon.core.JsonReaderImpl.parseObject(JsonReaderImpl.java:181)
>> >> at
>> >>
>> org.apache.johnzon.core.JsonReaderImpl.readValue(JsonReaderImpl.java:82)
>> >> at
>> org.apache.johnzon.core.JsonReaderImpl.read(JsonReaderImpl.java:61)
>> >> at
>> >>
>> org.apache.johnzon.core.JsonReaderImpl.readObject(JsonReaderImpl.java:150)"
>> >>
>> >>
>> >> ....but if I replace all the backslashes in that file with a standard
>> >> character like "a", I get the error message that is probably intended
>> for
>> >> the above case:
>> >>
>> >> Too many characters. Maximum string/number length of 8192 exceeded on
>> >> [lineNumber=1, columnNumber=10591, streamOffset=10590]. Maybe increase
>> >> org.apache.johnzon.max-string-length in jsonp factory properties or
>> system
>> >> properties.
>> >>
>> >>
>> >> I think there's a bug where this message doesn't appear if when there
>> are
>> >> a lot of escape characters (slashes) that bring the string length over
>> the
>> >> limit. Any reason this proper warning wouldn't appear when triggered by
>> >> escape characters? This is occurring both on Windows and Unix.
>> >>
>> >> Thanks,
>> >> -Kean
>> >>
>>
>>
Re: JsonReader.readObject() fails with ArrayIndexOutOfBoundsException
when a string is too long, but is full of backslashes?
Posted by Kean Erickson <ke...@gmail.com>.
I swapped to 7.1.0 and am seeing the same thing. But I think I have a
better idea of what is going on by default when this property isn't
configured manually.
In JsonParserFactoryImpl, a system level maxSize value is set and to
default to 262144 if the max-string-length johnzon property isn't set as a
system property. But before this, CxfRSService sets this in a system
property as 8192. So the larger value 262144 never applies.
On Tue, Feb 26, 2019 at 1:44 PM Mark Struberg <st...@yahoo.de.invalid>
wrote:
> We did tons of fixes in Johnzon. Can you please try out TomEE-7.0.5? This
> version contains a much newer version of Johnzon.
>
> LieGrue,
> strub
>
>
> > Am 26.02.2019 um 20:54 schrieb Kean Erickson <ke...@gmail.com>:
> >
> > Sorry to reply to my own email but one more question on that--if I am to
> > set "org.apache.johnzon.max-string-length", is there anywhere that the
> > default value for this is documented? The error message claims the
> default
> > is 8192, but if I override it in system.properties with 4092 the warning
> > goes away even though that would have resulted in the limit being lowered
> > if the "Too many characters" error message were correct.. So I'm not
> really
> > sure what's going on.
> >
> > On Mon, Feb 25, 2019 at 5:19 PM Kean Erickson <ke...@gmail.com>
> > wrote:
> >
> >> I'm using TomEE webprofile 7.04 in an exploded WAR deployment. I need to
> >> use JsonReader's readObject() to read from a JsonReader created from a
> >> string, but the JsonReader's readObject() call fails ineloquently when
> the
> >> object read is too long. I unfortunately haven't been able to find a
> way to
> >> make a unit test that reproduces this, it always passes even if the
> string
> >> length is over this limit.. I think being scoped to a request may have
> >> something to do with it.
> >>
> >> After putting the attached bad.json in my user directory, I acquire the
> >> attached file's contents as such:
> >>
> >> File bad = new File(System.getProperty("user.dir")+"/bad.json");
> >> byte[] badbytes = Files.readAllBytes(bad.toPath());
> >> String badStr = new String(badbytes, StandardCharsets.UTF_8);
> >> try(JsonReader jr = Json.createReader(new StringReader(badStr))) {
> >> JsonObject json = jr.readObject();
> >> }
> >>
> >> From this I get the error:
> >>
> >> "java.lang.ArrayIndexOutOfBoundsException: 8192
> >> at
> >>
> org.apache.johnzon.core.JsonStreamParserImpl.appendToCopyBuffer(JsonStreamParserImpl.java:156)
> >> at
> >>
> org.apache.johnzon.core.JsonStreamParserImpl.readString(JsonStreamParserImpl.java:581)
> >> at
> >>
> org.apache.johnzon.core.JsonStreamParserImpl.handleQuote(JsonStreamParserImpl.java:684)
> >> at
> >>
> org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:429)
> >> at
> >>
> org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:389)
> >> at
> >>
> org.apache.johnzon.core.JsonReaderImpl.parseObject(JsonReaderImpl.java:181)
> >> at
> >> org.apache.johnzon.core.JsonReaderImpl.readValue(JsonReaderImpl.java:82)
> >> at
> org.apache.johnzon.core.JsonReaderImpl.read(JsonReaderImpl.java:61)
> >> at
> >>
> org.apache.johnzon.core.JsonReaderImpl.readObject(JsonReaderImpl.java:150)"
> >>
> >>
> >> ....but if I replace all the backslashes in that file with a standard
> >> character like "a", I get the error message that is probably intended
> for
> >> the above case:
> >>
> >> Too many characters. Maximum string/number length of 8192 exceeded on
> >> [lineNumber=1, columnNumber=10591, streamOffset=10590]. Maybe increase
> >> org.apache.johnzon.max-string-length in jsonp factory properties or
> system
> >> properties.
> >>
> >>
> >> I think there's a bug where this message doesn't appear if when there
> are
> >> a lot of escape characters (slashes) that bring the string length over
> the
> >> limit. Any reason this proper warning wouldn't appear when triggered by
> >> escape characters? This is occurring both on Windows and Unix.
> >>
> >> Thanks,
> >> -Kean
> >>
>
>
Re: JsonReader.readObject() fails with ArrayIndexOutOfBoundsException
when a string is too long, but is full of backslashes?
Posted by Mark Struberg <st...@yahoo.de.INVALID>.
We did tons of fixes in Johnzon. Can you please try out TomEE-7.0.5? This version contains a much newer version of Johnzon.
LieGrue,
strub
> Am 26.02.2019 um 20:54 schrieb Kean Erickson <ke...@gmail.com>:
>
> Sorry to reply to my own email but one more question on that--if I am to
> set "org.apache.johnzon.max-string-length", is there anywhere that the
> default value for this is documented? The error message claims the default
> is 8192, but if I override it in system.properties with 4092 the warning
> goes away even though that would have resulted in the limit being lowered
> if the "Too many characters" error message were correct.. So I'm not really
> sure what's going on.
>
> On Mon, Feb 25, 2019 at 5:19 PM Kean Erickson <ke...@gmail.com>
> wrote:
>
>> I'm using TomEE webprofile 7.04 in an exploded WAR deployment. I need to
>> use JsonReader's readObject() to read from a JsonReader created from a
>> string, but the JsonReader's readObject() call fails ineloquently when the
>> object read is too long. I unfortunately haven't been able to find a way to
>> make a unit test that reproduces this, it always passes even if the string
>> length is over this limit.. I think being scoped to a request may have
>> something to do with it.
>>
>> After putting the attached bad.json in my user directory, I acquire the
>> attached file's contents as such:
>>
>> File bad = new File(System.getProperty("user.dir")+"/bad.json");
>> byte[] badbytes = Files.readAllBytes(bad.toPath());
>> String badStr = new String(badbytes, StandardCharsets.UTF_8);
>> try(JsonReader jr = Json.createReader(new StringReader(badStr))) {
>> JsonObject json = jr.readObject();
>> }
>>
>> From this I get the error:
>>
>> "java.lang.ArrayIndexOutOfBoundsException: 8192
>> at
>> org.apache.johnzon.core.JsonStreamParserImpl.appendToCopyBuffer(JsonStreamParserImpl.java:156)
>> at
>> org.apache.johnzon.core.JsonStreamParserImpl.readString(JsonStreamParserImpl.java:581)
>> at
>> org.apache.johnzon.core.JsonStreamParserImpl.handleQuote(JsonStreamParserImpl.java:684)
>> at
>> org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:429)
>> at
>> org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:389)
>> at
>> org.apache.johnzon.core.JsonReaderImpl.parseObject(JsonReaderImpl.java:181)
>> at
>> org.apache.johnzon.core.JsonReaderImpl.readValue(JsonReaderImpl.java:82)
>> at org.apache.johnzon.core.JsonReaderImpl.read(JsonReaderImpl.java:61)
>> at
>> org.apache.johnzon.core.JsonReaderImpl.readObject(JsonReaderImpl.java:150)"
>>
>>
>> ....but if I replace all the backslashes in that file with a standard
>> character like "a", I get the error message that is probably intended for
>> the above case:
>>
>> Too many characters. Maximum string/number length of 8192 exceeded on
>> [lineNumber=1, columnNumber=10591, streamOffset=10590]. Maybe increase
>> org.apache.johnzon.max-string-length in jsonp factory properties or system
>> properties.
>>
>>
>> I think there's a bug where this message doesn't appear if when there are
>> a lot of escape characters (slashes) that bring the string length over the
>> limit. Any reason this proper warning wouldn't appear when triggered by
>> escape characters? This is occurring both on Windows and Unix.
>>
>> Thanks,
>> -Kean
>>
Re: JsonReader.readObject() fails with ArrayIndexOutOfBoundsException
when a string is too long, but is full of backslashes?
Posted by Kean Erickson <ke...@gmail.com>.
Sorry to reply to my own email but one more question on that--if I am to
set "org.apache.johnzon.max-string-length", is there anywhere that the
default value for this is documented? The error message claims the default
is 8192, but if I override it in system.properties with 4092 the warning
goes away even though that would have resulted in the limit being lowered
if the "Too many characters" error message were correct.. So I'm not really
sure what's going on.
On Mon, Feb 25, 2019 at 5:19 PM Kean Erickson <ke...@gmail.com>
wrote:
> I'm using TomEE webprofile 7.04 in an exploded WAR deployment. I need to
> use JsonReader's readObject() to read from a JsonReader created from a
> string, but the JsonReader's readObject() call fails ineloquently when the
> object read is too long. I unfortunately haven't been able to find a way to
> make a unit test that reproduces this, it always passes even if the string
> length is over this limit.. I think being scoped to a request may have
> something to do with it.
>
> After putting the attached bad.json in my user directory, I acquire the
> attached file's contents as such:
>
> File bad = new File(System.getProperty("user.dir")+"/bad.json");
> byte[] badbytes = Files.readAllBytes(bad.toPath());
> String badStr = new String(badbytes, StandardCharsets.UTF_8);
> try(JsonReader jr = Json.createReader(new StringReader(badStr))) {
> JsonObject json = jr.readObject();
> }
>
> From this I get the error:
>
> "java.lang.ArrayIndexOutOfBoundsException: 8192
> at
> org.apache.johnzon.core.JsonStreamParserImpl.appendToCopyBuffer(JsonStreamParserImpl.java:156)
> at
> org.apache.johnzon.core.JsonStreamParserImpl.readString(JsonStreamParserImpl.java:581)
> at
> org.apache.johnzon.core.JsonStreamParserImpl.handleQuote(JsonStreamParserImpl.java:684)
> at
> org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:429)
> at
> org.apache.johnzon.core.JsonStreamParserImpl.next(JsonStreamParserImpl.java:389)
> at
> org.apache.johnzon.core.JsonReaderImpl.parseObject(JsonReaderImpl.java:181)
> at
> org.apache.johnzon.core.JsonReaderImpl.readValue(JsonReaderImpl.java:82)
> at org.apache.johnzon.core.JsonReaderImpl.read(JsonReaderImpl.java:61)
> at
> org.apache.johnzon.core.JsonReaderImpl.readObject(JsonReaderImpl.java:150)"
>
>
> ....but if I replace all the backslashes in that file with a standard
> character like "a", I get the error message that is probably intended for
> the above case:
>
> Too many characters. Maximum string/number length of 8192 exceeded on
> [lineNumber=1, columnNumber=10591, streamOffset=10590]. Maybe increase
> org.apache.johnzon.max-string-length in jsonp factory properties or system
> properties.
>
>
> I think there's a bug where this message doesn't appear if when there are
> a lot of escape characters (slashes) that bring the string length over the
> limit. Any reason this proper warning wouldn't appear when triggered by
> escape characters? This is occurring both on Windows and Unix.
>
> Thanks,
> -Kean
>