You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Johan Compagner <jc...@servoy.com> on 2013/11/16 15:09:31 UTC
setting the text or binary buffer size for websockets
Hi
i read this:
http://tomcat.apache.org/tomcat-7.0-doc/web-socket-howto.html
so what i do is add this into the web.xml
--
Johan Compagner
Servoy
Re: setting the text or binary buffer size for websockets
Posted by Igor Urisman <ig...@gmail.com>.
Johan,
What you've described is exactly what works for me. But I am still on RC1.
-Igor.
On Sat, Nov 16, 2013 at 6:12 AM, Johan Compagner <jc...@servoy.com>wrote:
> sorry, mail did go to soon...
>
> I do this in the web.xml (directly in the web-app tag)
>
> <context-param>
> <param-name>org.apache.tomcat.websocket.textBufferSize</param-name>
> <param-value>32768</param-value>
> </context-param>
> <context-param>
> <param-name>org.apache.tomcat.websocket.binaryBufferSize</param-name>
> <param-value>32768</param-value>
> </context-param>
>
> But this doesn't seem to have any effect, i still see in the browser stuff
> like frames of max 8192 (and continuation frames)
>
> We have problems (with chrome) with all kinds of errors when sending these
> frames (invalid opcode, utf char encoding problem, reserved words 1 ,2 ,3
> errors in the browser)
> So i want to see if i just don't use frames what the result is then
>
>
> Johan
>
>
> On 16 November 2013 15:09, Johan Compagner <jc...@servoy.com> wrote:
>
> > Hi
> >
> > i read this:
> >
> > http://tomcat.apache.org/tomcat-7.0-doc/web-socket-howto.html
> >
> > so what i do is add this into the web.xml
> >
> > --
> > Johan Compagner
> > Servoy
> >
>
>
>
> --
> Johan Compagner
> Servoy
>
Re: setting the text or binary buffer size for websockets
Posted by Johan Compagner <jc...@servoy.com>.
Konstantin,
> Right, I also think the calls to getBasicRemote().sendText(...) should
> be synchronized as multiple threads can concurrently access this method for
> a particular client (that however could mean that if one client stops
> receiving from its WebSocket connection, no other client can receive
> messages - to solve this, e.g. one could use a dedicated thread for each
> client which takes and sends messages from a queue, or use the
> RemoteEndpoint.Async).
> Oracle's javadoc of RemoteEndpoint.Basic says:
>
> "If the websocket connection underlying this RemoteEndpoint is busy
> sending a message when a call is made to send another one, for example if
> two threads attempt to call a send method concurrently, or if a developer
> attempts to send a new message while in the middle of sending an existing
> one, the send method called while the connection is already busy may throw
> an IllegalStateException."
>
ah ok, that makes sense then, i still would say why not make sendText of
BasicRemote a synchronized method inside tomcat.
Because that is what it really needs to be anyway, but doing a sync around
that is fine in our code.
But mistakes in this area are easily made, see for example the chat
example. That one is not really thread safe, i guess the private static
void broadcast(String msg) method should really sync around
the connections field.
And then i guess that connections doesn't have to be a copyonwritelist
(also sync on it when changes to that list are made)
What i still don't understand then when, had exactly the same code and
instead of getBasicRemote() i do getAsyncRemote() that i also did get the
same problems
A sync in my code wouldn't fix that at all, because the method returns
right away..
So i guess this has to be fixed by tomcat itself? So that asyncremote does
take care of of the synchronization?
> However, even after adding synchronization, I get stability issues and
> several exceptions on current Tomcat 8 trunk when sending large messages. I
> have filed a report here (I don't know if these are the same issues that
> you got): https://issues.apache.org/bugzilla/show_bug.cgi?id=55799
>
>
>
thats quite the same stuff, i only did get some other reported stuff inside
the browser, i guess because of the scrambled data.
I didn't really see that illegalstate exception showing up in the log.
johan
RE: setting the text or binary buffer size for websockets
Posted by Konstantin Preißer <kp...@apache.org>.
Hi Johan,
> -----Original Message-----
> From: Johan Compagner [mailto:jcompagner@servoy.com]
> Sent: Tuesday, November 19, 2013 6:13 PM
> To: Tomcat Users List
> Subject: Re: setting the text or binary buffer size for websockets
>
> >
> > > I expect that i can send now 32K at
> > > once of text (or binary) withing that "continuation frame"
> >
> > The buffer sizes control the input buffer - i.e. they control the
> > maximum size of a message that can be received if an application doesn't
> > support partial messages.
> >
> > The output buffers are all 8K.
> >
> > The splitting of a WebSocket message into multiple frames should be
> > transparent to the application.
> >
> >
>
> ah thx.
>
> But then i think a doc needs to be updated:
>
> http://tomcat.apache.org/tomcat-8.0-doc/web-socket-howto.html
>
> that doesn't tell me at all that it only input buffers, the property name
> doesn't say that and also the text doesn't mention that at all
> it just says: "The default buffer size for text messages is 8192 bytes" and
> many times in java that means an output buffer.
>
> But i did some more testing, and i think the ChatAnnotation should be
> altered as an example, because now it just does this:
>
> private static void broadcast(String msg) {
> for (ChatAnnotation client : connections) {
> try {
> client.session.getBasicRemote().sendText(msg);
>
>
> So without any synchronization it sends text..
> And this can happen in many threads (depending on which chat client makes
> the message)
> But i notice now that we have to sync around that line. If i don't do that
> then when sending large message (that will be split up in frames) funny
> stuff happens with a lot of weird exceptions on the browser side
>
> I am not 100% sure yet, still testing but it seems that if i sync around
> .getBasicRemote().sendText(msg); that it will work out fine
> But if i don't do that it doesn't work.
Right, I also think the calls to getBasicRemote().sendText(...) should be synchronized as multiple threads can concurrently access this method for a particular client (that however could mean that if one client stops receiving from its WebSocket connection, no other client can receive messages - to solve this, e.g. one could use a dedicated thread for each client which takes and sends messages from a queue, or use the RemoteEndpoint.Async).
Oracle's javadoc of RemoteEndpoint.Basic says:
"If the websocket connection underlying this RemoteEndpoint is busy sending a message when a call is made to send another one, for example if two threads attempt to call a send method concurrently, or if a developer attempts to send a new message while in the middle of sending an existing one, the send method called while the connection is already busy may throw an IllegalStateException."
However, even after adding synchronization, I get stability issues and several exceptions on current Tomcat 8 trunk when sending large messages. I have filed a report here (I don't know if these are the same issues that you got): https://issues.apache.org/bugzilla/show_bug.cgi?id=55799
Regards,
Konstantin Preißer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: setting the text or binary buffer size for websockets
Posted by Mark Thomas <ma...@apache.org>.
On 19/11/2013 17:12, Johan Compagner wrote:
>>
>>> I expect that i can send now 32K at
>>> once of text (or binary) withing that "continuation frame"
>>
>> The buffer sizes control the input buffer - i.e. they control the
>> maximum size of a message that can be received if an application doesn't
>> support partial messages.
>>
>> The output buffers are all 8K.
>>
>> The splitting of a WebSocket message into multiple frames should be
>> transparent to the application.
>>
>>
>
> ah thx.
>
> But then i think a doc needs to be updated:
>
> http://tomcat.apache.org/tomcat-8.0-doc/web-socket-howto.html
>
> that doesn't tell me at all that it only input buffers, the property name
> doesn't say that and also the text doesn't mention that at all
> it just says: "The default buffer size for text messages is 8192 bytes" and
> many times in java that means an output buffer.
Done.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: setting the text or binary buffer size for websockets
Posted by Johan Compagner <jc...@servoy.com>.
>
> > I expect that i can send now 32K at
> > once of text (or binary) withing that "continuation frame"
>
> The buffer sizes control the input buffer - i.e. they control the
> maximum size of a message that can be received if an application doesn't
> support partial messages.
>
> The output buffers are all 8K.
>
> The splitting of a WebSocket message into multiple frames should be
> transparent to the application.
>
>
ah thx.
But then i think a doc needs to be updated:
http://tomcat.apache.org/tomcat-8.0-doc/web-socket-howto.html
that doesn't tell me at all that it only input buffers, the property name
doesn't say that and also the text doesn't mention that at all
it just says: "The default buffer size for text messages is 8192 bytes" and
many times in java that means an output buffer.
But i did some more testing, and i think the ChatAnnotation should be
altered as an example, because now it just does this:
private static void broadcast(String msg) {
for (ChatAnnotation client : connections) {
try {
client.session.getBasicRemote().sendText(msg);
So without any synchronization it sends text..
And this can happen in many threads (depending on which chat client makes
the message)
But i notice now that we have to sync around that line. If i don't do that
then when sending large message (that will be split up in frames) funny
stuff happens with a lot of weird exceptions on the browser side
I am not 100% sure yet, still testing but it seems that if i sync around
.getBasicRemote().sendText(msg); that it will work out fine
But if i don't do that it doesn't work.
Is that logical? even getAsyncRemote doesn't work either.. (this is for me
very weird,i would have expected that the async remote would solve this
problem anyway)
johan
Re: setting the text or binary buffer size for websockets
Posted by Mark Thomas <ma...@apache.org>.
On 19/11/2013 14:15, Johan Compagner wrote:
> If i run that (http://localhost:8080/examples/websocket/chat.xhtml)
>
> Type in a string that will go over the 8K boundary
> Then in chrome it will still display a frame of 8K and then "continuation
> frame (Opcode 0)" which is the rest.
>
> am i expecting the wrong thing here?
Yes.
> I expect that i can send now 32K at
> once of text (or binary) withing that "continuation frame"
The buffer sizes control the input buffer - i.e. they control the
maximum size of a message that can be received if an application doesn't
support partial messages.
The output buffers are all 8K.
The splitting of a WebSocket message into multiple frames should be
transparent to the application.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: setting the text or binary buffer size for websockets
Posted by Johan Compagner <jc...@servoy.com>.
Hi
I have just tested it more, now just with the examples tomcat ships (the
chat example)
What i first did is add in web.xml these lines:
<context-param>
<param-name>org.apache.tomcat.websocket.textBufferSize</param-name>
<param-value>32768</param-value>
</context-param>
<context-param>
<param-name>org.apache.tomcat.websocket.binaryBufferSize</param-name>
<param-value>32768</param-value>
</context-param>
(right after metadata-complete="true"> so right at the beginning of the
web-app tag)
then i changed the ChatAnnotation class, in the incoming method i added 3
lines of code:
filteredMessage =
filteredMessage+filteredMessage+filteredMessage+filteredMessage+filteredMessage+filteredMessage+filteredMessage;
filteredMessage =
filteredMessage+filteredMessage+filteredMessage+filteredMessage+filteredMessage+filteredMessage+filteredMessage;
filteredMessage =
filteredMessage+filteredMessage+filteredMessage+filteredMessage+filteredMessage+filteredMessage+filteredMessage;
so that the message send back will be large
If i run that (http://localhost:8080/examples/websocket/chat.xhtml)
Type in a string that will go over the 8K boundary
Then in chrome it will still display a frame of 8K and then "continuation
frame (Opcode 0)" which is the rest.
am i expecting the wrong thing here? I expect that i can send now 32K at
once of text (or binary) withing that "continuation frame"
Johan
On 19 November 2013 09:38, Johan Compagner <jc...@servoy.com> wrote:
>
>
>
> On 19 November 2013 03:55, Igor Urisman <ig...@gmail.com> wrote:
>
>> Upgraded my environment to 8RC5 and this feature works for me.
>> Don't know how much help this is, but here's my deployment descriptor:
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>>
>> <web-app xmlns="http://java.sun.com/xml/ns/javaee"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
>> http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
>> version="3.0">
>>
>> <display-name>FERMI Framework Test Application</display-name>
>>
>> <context-param>
>> <param-name>org.apache.tomcat.websocket.textBufferSize</param-name>
>> <param-value>10240</param-value>
>> </context-param>
>>
>> </web-app>
>>
>>
> So if you do that, and you send large stuff to the browser then for
> example in Chrome in the network tab
> you do see on the websocket connection that the frames are bigger then 8K
> ? So in your example the frames are break up in 10k?
>
>
> johan
>
--
Johan Compagner
Servoy
Re: setting the text or binary buffer size for websockets
Posted by Johan Compagner <jc...@servoy.com>.
On 19 November 2013 03:55, Igor Urisman <ig...@gmail.com> wrote:
> Upgraded my environment to 8RC5 and this feature works for me.
> Don't know how much help this is, but here's my deployment descriptor:
>
> <?xml version="1.0" encoding="UTF-8"?>
>
> <web-app xmlns="http://java.sun.com/xml/ns/javaee"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
> http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
> version="3.0">
>
> <display-name>FERMI Framework Test Application</display-name>
>
> <context-param>
> <param-name>org.apache.tomcat.websocket.textBufferSize</param-name>
> <param-value>10240</param-value>
> </context-param>
>
> </web-app>
>
>
So if you do that, and you send large stuff to the browser then for example
in Chrome in the network tab
you do see on the websocket connection that the frames are bigger then 8K ?
So in your example the frames are break up in 10k?
johan
Re: setting the text or binary buffer size for websockets
Posted by Igor Urisman <ig...@gmail.com>.
Upgraded my environment to 8RC5 and this feature works for me.
Don't know how much help this is, but here's my deployment descriptor:
<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
version="3.0">
<display-name>FERMI Framework Test Application</display-name>
<context-param>
<param-name>org.apache.tomcat.websocket.textBufferSize</param-name>
<param-value>10240</param-value>
</context-param>
</web-app>
-Igor.
On Sun, Nov 17, 2013 at 7:15 AM, Johan Compagner <jc...@servoy.com>wrote:
> >
> >
> > >> Exactly which version of Tomcat 7 are you using?
> > >>
> > >>
> > > currently testing it on 8 RC5
> > >
> > > I can test on 7, but i guess thats the +/- the same code?
> >
> > Latest 8 RC is fine but then why are you looking at the Tomcat 7 docs?
> >
> >
> >
> first that i found, and also this
> http://tomcat.apache.org/tomcat-8.0-doc/web-socket-howto.html is the same
> i will look at it a bit more then, but for my current tests it didn't have
> any effect as far as i could see.
>
Re: setting the text or binary buffer size for websockets
Posted by Johan Compagner <jc...@servoy.com>.
>
>
> >> Exactly which version of Tomcat 7 are you using?
> >>
> >>
> > currently testing it on 8 RC5
> >
> > I can test on 7, but i guess thats the +/- the same code?
>
> Latest 8 RC is fine but then why are you looking at the Tomcat 7 docs?
>
>
>
first that i found, and also this
http://tomcat.apache.org/tomcat-8.0-doc/web-socket-howto.html is the same
i will look at it a bit more then, but for my current tests it didn't have
any effect as far as i could see.
Re: setting the text or binary buffer size for websockets
Posted by Mark Thomas <ma...@apache.org>.
On 17/11/2013 13:09, Johan Compagner wrote:
> On 17 November 2013 12:55, Mark Thomas <ma...@apache.org> wrote:
>
>> On 16/11/2013 14:12, Johan Compagner wrote:
>>
>>> We have problems (with chrome) with all kinds of errors when sending
>> these
>>> frames (invalid opcode, utf char encoding problem, reserved words 1 ,2 ,3
>>> errors in the browser)
>>> So i want to see if i just don't use frames what the result is then
>>
>> Exactly which version of Tomcat 7 are you using?
>>
>>
> currently testing it on 8 RC5
>
> I can test on 7, but i guess thats the +/- the same code?
Latest 8 RC is fine but then why are you looking at the Tomcat 7 docs?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: setting the text or binary buffer size for websockets
Posted by Johan Compagner <jc...@servoy.com>.
On 17 November 2013 12:55, Mark Thomas <ma...@apache.org> wrote:
> On 16/11/2013 14:12, Johan Compagner wrote:
>
> > We have problems (with chrome) with all kinds of errors when sending
> these
> > frames (invalid opcode, utf char encoding problem, reserved words 1 ,2 ,3
> > errors in the browser)
> > So i want to see if i just don't use frames what the result is then
>
> Exactly which version of Tomcat 7 are you using?
>
>
currently testing it on 8 RC5
I can test on 7, but i guess thats the +/- the same code?
Re: setting the text or binary buffer size for websockets
Posted by Mark Thomas <ma...@apache.org>.
On 16/11/2013 14:12, Johan Compagner wrote:
> We have problems (with chrome) with all kinds of errors when sending these
> frames (invalid opcode, utf char encoding problem, reserved words 1 ,2 ,3
> errors in the browser)
> So i want to see if i just don't use frames what the result is then
Exactly which version of Tomcat 7 are you using?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: setting the text or binary buffer size for websockets
Posted by Johan Compagner <jc...@servoy.com>.
sorry, mail did go to soon...
I do this in the web.xml (directly in the web-app tag)
<context-param>
<param-name>org.apache.tomcat.websocket.textBufferSize</param-name>
<param-value>32768</param-value>
</context-param>
<context-param>
<param-name>org.apache.tomcat.websocket.binaryBufferSize</param-name>
<param-value>32768</param-value>
</context-param>
But this doesn't seem to have any effect, i still see in the browser stuff
like frames of max 8192 (and continuation frames)
We have problems (with chrome) with all kinds of errors when sending these
frames (invalid opcode, utf char encoding problem, reserved words 1 ,2 ,3
errors in the browser)
So i want to see if i just don't use frames what the result is then
Johan
On 16 November 2013 15:09, Johan Compagner <jc...@servoy.com> wrote:
> Hi
>
> i read this:
>
> http://tomcat.apache.org/tomcat-7.0-doc/web-socket-howto.html
>
> so what i do is add this into the web.xml
>
> --
> Johan Compagner
> Servoy
>
--
Johan Compagner
Servoy