You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@wookie.apache.org by Ross Gardler <rg...@apache.org> on 2010/10/25 11:44:22 UTC
Node.js vs DWR Re: [jira] Commented: (WOOKIE-155) Experimental setup for shared data using websockets, node.js and redis
Scott,
I've not run this experimental code, however, I am following your work. It sounds really interesting. I confess to not knowing much about either DWR or node.js (other than the latter is attracting lots of attention).
What strikes me from your comments on thus work is that it seems to be z no-brained. But since you've opted to go with RTC on this I'm wondering what the potential drawback is. Why not just commit it? What is your primary concern?
Note I'm not suggesting we commit the code now, so close to a release, but perhaps immediately after the release?
With respect to performance issues, maybe we should make it a priority to do some profiling for the next release?
If we do add profiling we could do the node.js in a branch and compare results before merge. That might help us see where poor Wookie performance is being masked by improved node.js performance.
Sent from my mobile device.
On 25 Oct 2010, at 09:07, "Scott Wilson (JIRA)" <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/WOOKIE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924490#action_12924490 ]
>
> Scott Wilson commented on WOOKIE-155:
> -------------------------------------
>
> I built the experimental functionality into a "NodeWave" feature that can be used instead of the existing DWR-based feature and tried it out with some widgets. I'm not surprised that using Node.js+Redis+WebSockets was very, very fast (duh!) but am a bit concerned that DWR+Comet is surprisingly slow.
>
> I wonder which part of Wookie might have a performance issue? It could be any one of Comet/DWR, Jetty, the Wookie server code itself, JPA or Derby.
>
>
>> Experimental setup for shared data using websockets, node.js and redis
>> ----------------------------------------------------------------------
>>
>> Key: WOOKIE-155
>> URL: https://issues.apache.org/jira/browse/WOOKIE-155
>> Project: Wookie
>> Issue Type: Improvement
>> Components: Server
>> Reporter: Scott Wilson
>> Priority: Minor
>> Attachments: chat_dwr_vid.swf, chat_node_vid.swf, wavenoderedis.zip
>>
>>
>> This is an experimental setup that simulates replacing the DWR-based functionality of Wookie's Wave API implementation with one using Node.js, WebSockets, and Redis. The key motivation behind this experiment is to see how much more responsive Wookie shared state widgets can be using a fast Websockets implementation instead of Comet on a typical Java server stack.
>> To try it out, you need to install Node.js and the SocketIO websockets implementation. You also need to run a Redis server. This file contains the server-side logic.
>> To run the example:
>> 1. Start your redis server on the default port using:
>> ./redis-server
>> 2. In the folder you unzipped the code into, type:
>> node server.js
>> 3. In your browser (Safari and Chrome work well) open each of the testx.html files. Test and Test2 share the same SharedDataKey whereas Test3.html does not. Type in key:value pairs in the text boxes to send deltas to the wave state, and see them updated in other "widget instances".
>> Note the example is incomplete as it only handles state, not viewer or participants
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
Re: Node.js vs DWR Re: [jira] Commented: (WOOKIE-155) Experimental setup for shared data using websockets, node.js and redis
Posted by Ross Gardler <rg...@apache.org>.
The more separation we have of features and core the better. Of course this needs to be balanced against short term pain for possible long term gain.
Sent from my mobile device.
On 27 Oct 2010, at 18:13, Scott Wilson <sc...@gmail.com> wrote:
> Another consideration here is the extent to which we want in future to decouple the core Wookie widget server from the Wave feature.
>
> Unlike other Feature extensions it has specific code in the core server - WaveAPI, WaveAPIImpl, SharedDataKey, Participant, the DWR servlet descriptor etc.
>
> I guess one thing the Node experiments have been useful for is identifying how we could more cleanly separate out the Wave feature from the core - even if it is implemented using DWR.
>
> The model I've used in the current iteration of my experimental work has been to push both the SharedDataKey and the Widget Viewer object into the instance preferences - a bit of a hack, but it means that the whole business of managing participants, shared data, notifications and so on is moved out of the Wookie server code and into the Wave implementation. In this case its Node.js, but it could just as easily be the current implementation factored out into a separate service.
>
> On 27 Oct 2010, at 12:16, Scott Wilson wrote:
>
>> On 27 Oct 2010, at 10:37, Bernhard Hoisl wrote:
>>
>>>> Odd, if I delete those lines it doesn't work at all on Safari, on Firefox it works for a few deltas before it stops working. This is using the default local Jetty/Derby setup. Are you testing this on Tomcat?
>>>
>>> No, I am using Wookie in standalone mode and tried it with Firefox (were it works). Strangely, Chrome and Opera didn't respond, too.
>>
>> I've been playing around with it some more, and noticed that while in Firefox the transfer encoding of the response for DWR is "chunked" (as you'd expect for comet) for Safari its "identity".
>
Re: Node.js vs DWR Re: [jira] Commented: (WOOKIE-155) Experimental setup for shared data using websockets, node.js and redis
Posted by Scott Wilson <sc...@gmail.com>.
Another consideration here is the extent to which we want in future to decouple the core Wookie widget server from the Wave feature.
Unlike other Feature extensions it has specific code in the core server - WaveAPI, WaveAPIImpl, SharedDataKey, Participant, the DWR servlet descriptor etc.
I guess one thing the Node experiments have been useful for is identifying how we could more cleanly separate out the Wave feature from the core - even if it is implemented using DWR.
The model I've used in the current iteration of my experimental work has been to push both the SharedDataKey and the Widget Viewer object into the instance preferences - a bit of a hack, but it means that the whole business of managing participants, shared data, notifications and so on is moved out of the Wookie server code and into the Wave implementation. In this case its Node.js, but it could just as easily be the current implementation factored out into a separate service.
On 27 Oct 2010, at 12:16, Scott Wilson wrote:
> On 27 Oct 2010, at 10:37, Bernhard Hoisl wrote:
>
>>> Odd, if I delete those lines it doesn't work at all on Safari, on Firefox it works for a few deltas before it stops working. This is using the default local Jetty/Derby setup. Are you testing this on Tomcat?
>>
>> No, I am using Wookie in standalone mode and tried it with Firefox (were it works). Strangely, Chrome and Opera didn't respond, too.
>
> I've been playing around with it some more, and noticed that while in Firefox the transfer encoding of the response for DWR is "chunked" (as you'd expect for comet) for Safari its "identity".
Re: Node.js vs DWR Re: [jira] Commented: (WOOKIE-155) Experimental setup for shared data using websockets, node.js and redis
Posted by Scott Wilson <sc...@gmail.com>.
On 27 Oct 2010, at 10:37, Bernhard Hoisl wrote:
>> Odd, if I delete those lines it doesn't work at all on Safari, on Firefox it works for a few deltas before it stops working. This is using the default local Jetty/Derby setup. Are you testing this on Tomcat?
>
> No, I am using Wookie in standalone mode and tried it with Firefox (were it works). Strangely, Chrome and Opera didn't respond, too.
I've been playing around with it some more, and noticed that while in Firefox the transfer encoding of the response for DWR is "chunked" (as you'd expect for comet) for Safari its "identity".
Re: Node.js vs DWR Re: [jira] Commented: (WOOKIE-155) Experimental
setup for shared data using websockets, node.js and redis
Posted by Bernhard Hoisl <be...@wu.ac.at>.
> Odd, if I delete those lines it doesn't work at all on Safari, on Firefox it works for a few deltas before it stops working. This is using the default local Jetty/Derby setup. Are you testing this on Tomcat?
No, I am using Wookie in standalone mode and tried it with Firefox (were
it works). Strangely, Chrome and Opera didn't respond, too.
>
>>
>>> More info at: http://directwebremoting.org/dwr/reverse-ajax/configuration.html
>>
>> I have set an interval of 2.5 sec for polling by inserting the following lines:
>>
>> <init-param>
>> <param-name>disconnectedTime</param-name>
>> <param-value>2500</param-value>
>> </init-param>
>>
>> Default poll rate is 5 sec.
>>
>>>>
>>>>
>>>>>
>>>>> 2. Running an additional two services is a pain, and shouldn't be done unless it really is strictly necessary. For example, with the code I've developed you end up running:
>>>>>
>>>>> - Node.js
>>>>> - Redis
>>>>> - Jetty
>>>>> - Derby
>>>>>
>>>>> ... with relational data stored in Derby and tuples stored in Redis. Its certainly a more complex deployment than we have at present.
>>>>>
>>>>> 3. Integrating participants is less trivial than state data, and would not necessarily have any performance benefit
>>>>>
>>>>> 4. The benefits may be entirely due to websockets vs comet, in which case a websockets implementation in DWR would be the easiest path. However, I haven't seen a lot of discussion of websockets on the DWR lists.
>>>>>
>>>>> Alternatively, Jetty has WebSockets support as of V7.0.1. However, I'm not aware of any implementation for Tomcat.
>>>>>
>>>>>> Note I'm not suggesting we commit the code now, so close to a release, but perhaps immediately after the release?
>>>>>>
>>>>>> With respect to performance issues, maybe we should make it a priority to do some profiling for the next release?
>>>>>
>>>>> Agreed, I'd be wary of jumping to conclusions until we've identified where performance issues occur.
>>>>>
>>>>>> If we do add profiling we could do the node.js in a branch and compare results before merge. That might help us see where poor Wookie performance is being masked by improved node.js performance.
>>>>>
>>>>> Sounds good to me.
>>>>>
>>>>>>
>>>>>> Sent from my mobile device.
>>>>>>
>>>>>> On 25 Oct 2010, at 09:07, "Scott Wilson (JIRA)"<ji...@apache.org> wrote:
>>>>>>
>>>>>>>
>>>>>>> [ https://issues.apache.org/jira/browse/WOOKIE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924490#action_12924490 ]
>>>>>>>
>>>>>>> Scott Wilson commented on WOOKIE-155:
>>>>>>> -------------------------------------
>>>>>>>
>>>>>>> I built the experimental functionality into a "NodeWave" feature that can be used instead of the existing DWR-based feature and tried it out with some widgets. I'm not surprised that using Node.js+Redis+WebSockets was very, very fast (duh!) but am a bit concerned that DWR+Comet is surprisingly slow.
>>>>>>>
>>>>>>> I wonder which part of Wookie might have a performance issue? It could be any one of Comet/DWR, Jetty, the Wookie server code itself, JPA or Derby.
>>>>>>>
>>>>>>>
>>>>>>>> Experimental setup for shared data using websockets, node.js and redis
>>>>>>>> ----------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Key: WOOKIE-155
>>>>>>>> URL: https://issues.apache.org/jira/browse/WOOKIE-155
>>>>>>>> Project: Wookie
>>>>>>>> Issue Type: Improvement
>>>>>>>> Components: Server
>>>>>>>> Reporter: Scott Wilson
>>>>>>>> Priority: Minor
>>>>>>>> Attachments: chat_dwr_vid.swf, chat_node_vid.swf, wavenoderedis.zip
>>>>>>>>
>>>>>>>>
>>>>>>>> This is an experimental setup that simulates replacing the DWR-based functionality of Wookie's Wave API implementation with one using Node.js, WebSockets, and Redis. The key motivation behind this experiment is to see how much more responsive Wookie shared state widgets can be using a fast Websockets implementation instead of Comet on a typical Java server stack.
>>>>>>>> To try it out, you need to install Node.js and the SocketIO websockets implementation. You also need to run a Redis server. This file contains the server-side logic.
>>>>>>>> To run the example:
>>>>>>>> 1. Start your redis server on the default port using:
>>>>>>>> ./redis-server
>>>>>>>> 2. In the folder you unzipped the code into, type:
>>>>>>>> node server.js
>>>>>>>> 3. In your browser (Safari and Chrome work well) open each of the testx.html files. Test and Test2 share the same SharedDataKey whereas Test3.html does not. Type in key:value pairs in the text boxes to send deltas to the wave state, and see them updated in other "widget instances".
>>>>>>>> Note the example is incomplete as it only handles state, not viewer or participants
>>>>>>>
>>>>>>> --
>>>>>>> This message is automatically generated by JIRA.
>>>>>>> -
>>>>>>> You can reply to this email to add a comment to the issue online.
>>>>>>>
>>>>>
>>>
>
Re: Node.js vs DWR Re: [jira] Commented: (WOOKIE-155) Experimental setup for shared data using websockets, node.js and redis
Posted by Scott Wilson <sc...@gmail.com>.
On 27 Oct 2010, at 07:58, Bernhard Hoisl wrote:
>> It *should* use Comet as the wave.js file calls:
>>
>> dwr.engine.setActiveReverseAjax(true);
>>
>> ... which should tell DWR to use Comet rather than polling.
>>
>> Plus DWR is configured in web.xml with:
>>
>> <init-param>
>> <param-name>activeReverseAjaxEnabled</param-name>
>> <param-value>true</param-value>
>> </init-param>
>
> For me, Reverse-Ajax is one thing, the communication mode is another. DWR is configured for Reverse-Ajax - clear to me. But, my widgets definitely are polling to receive updates from Wookie (and I haven't touched the significant parts in web.xml, so I just use it as delivered with Wookie). If I open Firebug I clearly see the polling requests.
>
> Every interval [WOOKIE]/wookie/dwr/call/plainpoll/ReverseAjax.dwr is called to see if there are new updates.
>
> In my web.xml there are also the lines
>
> <init-param>
> <param-name>org.directwebremoting.extend.ServerLoadMonitor</param-name>
> <param-value>org.directwebremoting.impl.PollingServerLoadMonitor</param-value>
> </init-param>
>
> which are telling DWR to use polling instead of, e.g. a long-held HTTP response (aka Comet).
>
> If I delete those lines Comet style is used and the communication is really fast (to me as fast as Node.js in you video). Again checking with Firebug I can see that now an HTTP response is held open. All this is described in the URL you mentioned below (see Polling Mode section):
Odd, if I delete those lines it doesn't work at all on Safari, on Firefox it works for a few deltas before it stops working. This is using the default local Jetty/Derby setup. Are you testing this on Tomcat?
>
>> More info at: http://directwebremoting.org/dwr/reverse-ajax/configuration.html
>
> I have set an interval of 2.5 sec for polling by inserting the following lines:
>
> <init-param>
> <param-name>disconnectedTime</param-name>
> <param-value>2500</param-value>
> </init-param>
>
> Default poll rate is 5 sec.
>
>>>
>>>
>>>>
>>>> 2. Running an additional two services is a pain, and shouldn't be done unless it really is strictly necessary. For example, with the code I've developed you end up running:
>>>>
>>>> - Node.js
>>>> - Redis
>>>> - Jetty
>>>> - Derby
>>>>
>>>> ... with relational data stored in Derby and tuples stored in Redis. Its certainly a more complex deployment than we have at present.
>>>>
>>>> 3. Integrating participants is less trivial than state data, and would not necessarily have any performance benefit
>>>>
>>>> 4. The benefits may be entirely due to websockets vs comet, in which case a websockets implementation in DWR would be the easiest path. However, I haven't seen a lot of discussion of websockets on the DWR lists.
>>>>
>>>> Alternatively, Jetty has WebSockets support as of V7.0.1. However, I'm not aware of any implementation for Tomcat.
>>>>
>>>>> Note I'm not suggesting we commit the code now, so close to a release, but perhaps immediately after the release?
>>>>>
>>>>> With respect to performance issues, maybe we should make it a priority to do some profiling for the next release?
>>>>
>>>> Agreed, I'd be wary of jumping to conclusions until we've identified where performance issues occur.
>>>>
>>>>> If we do add profiling we could do the node.js in a branch and compare results before merge. That might help us see where poor Wookie performance is being masked by improved node.js performance.
>>>>
>>>> Sounds good to me.
>>>>
>>>>>
>>>>> Sent from my mobile device.
>>>>>
>>>>> On 25 Oct 2010, at 09:07, "Scott Wilson (JIRA)"<ji...@apache.org> wrote:
>>>>>
>>>>>>
>>>>>> [ https://issues.apache.org/jira/browse/WOOKIE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924490#action_12924490 ]
>>>>>>
>>>>>> Scott Wilson commented on WOOKIE-155:
>>>>>> -------------------------------------
>>>>>>
>>>>>> I built the experimental functionality into a "NodeWave" feature that can be used instead of the existing DWR-based feature and tried it out with some widgets. I'm not surprised that using Node.js+Redis+WebSockets was very, very fast (duh!) but am a bit concerned that DWR+Comet is surprisingly slow.
>>>>>>
>>>>>> I wonder which part of Wookie might have a performance issue? It could be any one of Comet/DWR, Jetty, the Wookie server code itself, JPA or Derby.
>>>>>>
>>>>>>
>>>>>>> Experimental setup for shared data using websockets, node.js and redis
>>>>>>> ----------------------------------------------------------------------
>>>>>>>
>>>>>>> Key: WOOKIE-155
>>>>>>> URL: https://issues.apache.org/jira/browse/WOOKIE-155
>>>>>>> Project: Wookie
>>>>>>> Issue Type: Improvement
>>>>>>> Components: Server
>>>>>>> Reporter: Scott Wilson
>>>>>>> Priority: Minor
>>>>>>> Attachments: chat_dwr_vid.swf, chat_node_vid.swf, wavenoderedis.zip
>>>>>>>
>>>>>>>
>>>>>>> This is an experimental setup that simulates replacing the DWR-based functionality of Wookie's Wave API implementation with one using Node.js, WebSockets, and Redis. The key motivation behind this experiment is to see how much more responsive Wookie shared state widgets can be using a fast Websockets implementation instead of Comet on a typical Java server stack.
>>>>>>> To try it out, you need to install Node.js and the SocketIO websockets implementation. You also need to run a Redis server. This file contains the server-side logic.
>>>>>>> To run the example:
>>>>>>> 1. Start your redis server on the default port using:
>>>>>>> ./redis-server
>>>>>>> 2. In the folder you unzipped the code into, type:
>>>>>>> node server.js
>>>>>>> 3. In your browser (Safari and Chrome work well) open each of the testx.html files. Test and Test2 share the same SharedDataKey whereas Test3.html does not. Type in key:value pairs in the text boxes to send deltas to the wave state, and see them updated in other "widget instances".
>>>>>>> Note the example is incomplete as it only handles state, not viewer or participants
>>>>>>
>>>>>> --
>>>>>> This message is automatically generated by JIRA.
>>>>>> -
>>>>>> You can reply to this email to add a comment to the issue online.
>>>>>>
>>>>
>>
Re: Node.js vs DWR Re: [jira] Commented: (WOOKIE-155) Experimental
setup for shared data using websockets, node.js and redis
Posted by Bernhard Hoisl <be...@wu.ac.at>.
> It *should* use Comet as the wave.js file calls:
>
> dwr.engine.setActiveReverseAjax(true);
>
> ... which should tell DWR to use Comet rather than polling.
>
> Plus DWR is configured in web.xml with:
>
> <init-param>
> <param-name>activeReverseAjaxEnabled</param-name>
> <param-value>true</param-value>
> </init-param>
For me, Reverse-Ajax is one thing, the communication mode is another.
DWR is configured for Reverse-Ajax - clear to me. But, my widgets
definitely are polling to receive updates from Wookie (and I haven't
touched the significant parts in web.xml, so I just use it as delivered
with Wookie). If I open Firebug I clearly see the polling requests.
Every interval [WOOKIE]/wookie/dwr/call/plainpoll/ReverseAjax.dwr is
called to see if there are new updates.
In my web.xml there are also the lines
<init-param>
<param-name>org.directwebremoting.extend.ServerLoadMonitor</param-name>
<param-value>org.directwebremoting.impl.PollingServerLoadMonitor</param-value>
</init-param>
which are telling DWR to use polling instead of, e.g. a long-held HTTP
response (aka Comet).
If I delete those lines Comet style is used and the communication is
really fast (to me as fast as Node.js in you video). Again checking with
Firebug I can see that now an HTTP response is held open. All this is
described in the URL you mentioned below (see Polling Mode section):
> More info at: http://directwebremoting.org/dwr/reverse-ajax/configuration.html
I have set an interval of 2.5 sec for polling by inserting the following
lines:
<init-param>
<param-name>disconnectedTime</param-name>
<param-value>2500</param-value>
</init-param>
Default poll rate is 5 sec.
>>
>>
>>>
>>> 2. Running an additional two services is a pain, and shouldn't be done unless it really is strictly necessary. For example, with the code I've developed you end up running:
>>>
>>> - Node.js
>>> - Redis
>>> - Jetty
>>> - Derby
>>>
>>> ... with relational data stored in Derby and tuples stored in Redis. Its certainly a more complex deployment than we have at present.
>>>
>>> 3. Integrating participants is less trivial than state data, and would not necessarily have any performance benefit
>>>
>>> 4. The benefits may be entirely due to websockets vs comet, in which case a websockets implementation in DWR would be the easiest path. However, I haven't seen a lot of discussion of websockets on the DWR lists.
>>>
>>> Alternatively, Jetty has WebSockets support as of V7.0.1. However, I'm not aware of any implementation for Tomcat.
>>>
>>>> Note I'm not suggesting we commit the code now, so close to a release, but perhaps immediately after the release?
>>>>
>>>> With respect to performance issues, maybe we should make it a priority to do some profiling for the next release?
>>>
>>> Agreed, I'd be wary of jumping to conclusions until we've identified where performance issues occur.
>>>
>>>> If we do add profiling we could do the node.js in a branch and compare results before merge. That might help us see where poor Wookie performance is being masked by improved node.js performance.
>>>
>>> Sounds good to me.
>>>
>>>>
>>>> Sent from my mobile device.
>>>>
>>>> On 25 Oct 2010, at 09:07, "Scott Wilson (JIRA)"<ji...@apache.org> wrote:
>>>>
>>>>>
>>>>> [ https://issues.apache.org/jira/browse/WOOKIE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924490#action_12924490 ]
>>>>>
>>>>> Scott Wilson commented on WOOKIE-155:
>>>>> -------------------------------------
>>>>>
>>>>> I built the experimental functionality into a "NodeWave" feature that can be used instead of the existing DWR-based feature and tried it out with some widgets. I'm not surprised that using Node.js+Redis+WebSockets was very, very fast (duh!) but am a bit concerned that DWR+Comet is surprisingly slow.
>>>>>
>>>>> I wonder which part of Wookie might have a performance issue? It could be any one of Comet/DWR, Jetty, the Wookie server code itself, JPA or Derby.
>>>>>
>>>>>
>>>>>> Experimental setup for shared data using websockets, node.js and redis
>>>>>> ----------------------------------------------------------------------
>>>>>>
>>>>>> Key: WOOKIE-155
>>>>>> URL: https://issues.apache.org/jira/browse/WOOKIE-155
>>>>>> Project: Wookie
>>>>>> Issue Type: Improvement
>>>>>> Components: Server
>>>>>> Reporter: Scott Wilson
>>>>>> Priority: Minor
>>>>>> Attachments: chat_dwr_vid.swf, chat_node_vid.swf, wavenoderedis.zip
>>>>>>
>>>>>>
>>>>>> This is an experimental setup that simulates replacing the DWR-based functionality of Wookie's Wave API implementation with one using Node.js, WebSockets, and Redis. The key motivation behind this experiment is to see how much more responsive Wookie shared state widgets can be using a fast Websockets implementation instead of Comet on a typical Java server stack.
>>>>>> To try it out, you need to install Node.js and the SocketIO websockets implementation. You also need to run a Redis server. This file contains the server-side logic.
>>>>>> To run the example:
>>>>>> 1. Start your redis server on the default port using:
>>>>>> ./redis-server
>>>>>> 2. In the folder you unzipped the code into, type:
>>>>>> node server.js
>>>>>> 3. In your browser (Safari and Chrome work well) open each of the testx.html files. Test and Test2 share the same SharedDataKey whereas Test3.html does not. Type in key:value pairs in the text boxes to send deltas to the wave state, and see them updated in other "widget instances".
>>>>>> Note the example is incomplete as it only handles state, not viewer or participants
>>>>>
>>>>> --
>>>>> This message is automatically generated by JIRA.
>>>>> -
>>>>> You can reply to this email to add a comment to the issue online.
>>>>>
>>>
>
Re: Node.js vs DWR Re: [jira] Commented: (WOOKIE-155) Experimental setup for shared data using websockets, node.js and redis
Posted by Scott Wilson <sc...@gmail.com>.
On 25 Oct 2010, at 12:19, Bernhard Hoisl wrote:
>> 1. We may simply have an issue with our configuration of DWR or Jetty, and we may be able bump the speed up to a more satisfactory level without major changes
>
> Maybe I misunderstood something but just by looking at the videos it seemed to me that DWR is using polling instead of comet style? I think to have seen that Wookie uses polling as standard configuration with an interval of 5 seconds. This would explain the long timeout between updates, but can be configured ...
>
> -Bernhard
It *should* use Comet as the wave.js file calls:
dwr.engine.setActiveReverseAjax(true);
... which should tell DWR to use Comet rather than polling.
Plus DWR is configured in web.xml with:
<init-param>
<param-name>activeReverseAjaxEnabled</param-name>
<param-value>true</param-value>
</init-param>
More info at: http://directwebremoting.org/dwr/reverse-ajax/configuration.html
However there is clearly something wrong with performance here.
>
>
>>
>> 2. Running an additional two services is a pain, and shouldn't be done unless it really is strictly necessary. For example, with the code I've developed you end up running:
>>
>> - Node.js
>> - Redis
>> - Jetty
>> - Derby
>>
>> ... with relational data stored in Derby and tuples stored in Redis. Its certainly a more complex deployment than we have at present.
>>
>> 3. Integrating participants is less trivial than state data, and would not necessarily have any performance benefit
>>
>> 4. The benefits may be entirely due to websockets vs comet, in which case a websockets implementation in DWR would be the easiest path. However, I haven't seen a lot of discussion of websockets on the DWR lists.
>>
>> Alternatively, Jetty has WebSockets support as of V7.0.1. However, I'm not aware of any implementation for Tomcat.
>>
>>> Note I'm not suggesting we commit the code now, so close to a release, but perhaps immediately after the release?
>>>
>>> With respect to performance issues, maybe we should make it a priority to do some profiling for the next release?
>>
>> Agreed, I'd be wary of jumping to conclusions until we've identified where performance issues occur.
>>
>>> If we do add profiling we could do the node.js in a branch and compare results before merge. That might help us see where poor Wookie performance is being masked by improved node.js performance.
>>
>> Sounds good to me.
>>
>>>
>>> Sent from my mobile device.
>>>
>>> On 25 Oct 2010, at 09:07, "Scott Wilson (JIRA)"<ji...@apache.org> wrote:
>>>
>>>>
>>>> [ https://issues.apache.org/jira/browse/WOOKIE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924490#action_12924490 ]
>>>>
>>>> Scott Wilson commented on WOOKIE-155:
>>>> -------------------------------------
>>>>
>>>> I built the experimental functionality into a "NodeWave" feature that can be used instead of the existing DWR-based feature and tried it out with some widgets. I'm not surprised that using Node.js+Redis+WebSockets was very, very fast (duh!) but am a bit concerned that DWR+Comet is surprisingly slow.
>>>>
>>>> I wonder which part of Wookie might have a performance issue? It could be any one of Comet/DWR, Jetty, the Wookie server code itself, JPA or Derby.
>>>>
>>>>
>>>>> Experimental setup for shared data using websockets, node.js and redis
>>>>> ----------------------------------------------------------------------
>>>>>
>>>>> Key: WOOKIE-155
>>>>> URL: https://issues.apache.org/jira/browse/WOOKIE-155
>>>>> Project: Wookie
>>>>> Issue Type: Improvement
>>>>> Components: Server
>>>>> Reporter: Scott Wilson
>>>>> Priority: Minor
>>>>> Attachments: chat_dwr_vid.swf, chat_node_vid.swf, wavenoderedis.zip
>>>>>
>>>>>
>>>>> This is an experimental setup that simulates replacing the DWR-based functionality of Wookie's Wave API implementation with one using Node.js, WebSockets, and Redis. The key motivation behind this experiment is to see how much more responsive Wookie shared state widgets can be using a fast Websockets implementation instead of Comet on a typical Java server stack.
>>>>> To try it out, you need to install Node.js and the SocketIO websockets implementation. You also need to run a Redis server. This file contains the server-side logic.
>>>>> To run the example:
>>>>> 1. Start your redis server on the default port using:
>>>>> ./redis-server
>>>>> 2. In the folder you unzipped the code into, type:
>>>>> node server.js
>>>>> 3. In your browser (Safari and Chrome work well) open each of the testx.html files. Test and Test2 share the same SharedDataKey whereas Test3.html does not. Type in key:value pairs in the text boxes to send deltas to the wave state, and see them updated in other "widget instances".
>>>>> Note the example is incomplete as it only handles state, not viewer or participants
>>>>
>>>> --
>>>> This message is automatically generated by JIRA.
>>>> -
>>>> You can reply to this email to add a comment to the issue online.
>>>>
>>
Re: Node.js vs DWR Re: [jira] Commented: (WOOKIE-155) Experimental
setup for shared data using websockets, node.js and redis
Posted by Bernhard Hoisl <be...@wu.ac.at>.
> 1. We may simply have an issue with our configuration of DWR or Jetty, and we may be able bump the speed up to a more satisfactory level without major changes
Maybe I misunderstood something but just by looking at the videos it
seemed to me that DWR is using polling instead of comet style? I think
to have seen that Wookie uses polling as standard configuration with an
interval of 5 seconds. This would explain the long timeout between
updates, but can be configured ...
-Bernhard
>
> 2. Running an additional two services is a pain, and shouldn't be done unless it really is strictly necessary. For example, with the code I've developed you end up running:
>
> - Node.js
> - Redis
> - Jetty
> - Derby
>
> ... with relational data stored in Derby and tuples stored in Redis. Its certainly a more complex deployment than we have at present.
>
> 3. Integrating participants is less trivial than state data, and would not necessarily have any performance benefit
>
> 4. The benefits may be entirely due to websockets vs comet, in which case a websockets implementation in DWR would be the easiest path. However, I haven't seen a lot of discussion of websockets on the DWR lists.
>
> Alternatively, Jetty has WebSockets support as of V7.0.1. However, I'm not aware of any implementation for Tomcat.
>
>> Note I'm not suggesting we commit the code now, so close to a release, but perhaps immediately after the release?
>>
>> With respect to performance issues, maybe we should make it a priority to do some profiling for the next release?
>
> Agreed, I'd be wary of jumping to conclusions until we've identified where performance issues occur.
>
>> If we do add profiling we could do the node.js in a branch and compare results before merge. That might help us see where poor Wookie performance is being masked by improved node.js performance.
>
> Sounds good to me.
>
>>
>> Sent from my mobile device.
>>
>> On 25 Oct 2010, at 09:07, "Scott Wilson (JIRA)"<ji...@apache.org> wrote:
>>
>>>
>>> [ https://issues.apache.org/jira/browse/WOOKIE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924490#action_12924490 ]
>>>
>>> Scott Wilson commented on WOOKIE-155:
>>> -------------------------------------
>>>
>>> I built the experimental functionality into a "NodeWave" feature that can be used instead of the existing DWR-based feature and tried it out with some widgets. I'm not surprised that using Node.js+Redis+WebSockets was very, very fast (duh!) but am a bit concerned that DWR+Comet is surprisingly slow.
>>>
>>> I wonder which part of Wookie might have a performance issue? It could be any one of Comet/DWR, Jetty, the Wookie server code itself, JPA or Derby.
>>>
>>>
>>>> Experimental setup for shared data using websockets, node.js and redis
>>>> ----------------------------------------------------------------------
>>>>
>>>> Key: WOOKIE-155
>>>> URL: https://issues.apache.org/jira/browse/WOOKIE-155
>>>> Project: Wookie
>>>> Issue Type: Improvement
>>>> Components: Server
>>>> Reporter: Scott Wilson
>>>> Priority: Minor
>>>> Attachments: chat_dwr_vid.swf, chat_node_vid.swf, wavenoderedis.zip
>>>>
>>>>
>>>> This is an experimental setup that simulates replacing the DWR-based functionality of Wookie's Wave API implementation with one using Node.js, WebSockets, and Redis. The key motivation behind this experiment is to see how much more responsive Wookie shared state widgets can be using a fast Websockets implementation instead of Comet on a typical Java server stack.
>>>> To try it out, you need to install Node.js and the SocketIO websockets implementation. You also need to run a Redis server. This file contains the server-side logic.
>>>> To run the example:
>>>> 1. Start your redis server on the default port using:
>>>> ./redis-server
>>>> 2. In the folder you unzipped the code into, type:
>>>> node server.js
>>>> 3. In your browser (Safari and Chrome work well) open each of the testx.html files. Test and Test2 share the same SharedDataKey whereas Test3.html does not. Type in key:value pairs in the text boxes to send deltas to the wave state, and see them updated in other "widget instances".
>>>> Note the example is incomplete as it only handles state, not viewer or participants
>>>
>>> --
>>> This message is automatically generated by JIRA.
>>> -
>>> You can reply to this email to add a comment to the issue online.
>>>
>
Re: Node.js vs DWR Re: [jira] Commented: (WOOKIE-155) Experimental setup for shared data using websockets, node.js and redis
Posted by Scott Wilson <sc...@gmail.com>.
On 25 Oct 2010, at 10:44, Ross Gardler wrote:
> Scott,
>
> I've not run this experimental code, however, I am following your work. It sounds really interesting. I confess to not knowing much about either DWR or node.js (other than the latter is attracting lots of attention).
DWR is the component Wookie uses to implement Comet (reverse Ajax). It maps Java APIs to JavaScript that is injected in the widget.
Node.js is a server that uses the Chrome JavaScript engine to compile and run server-side JavaScript applications using a novel event-driven IO model. Its really fast, but quite low-level. Its also very new (I think it started in February) and has few production deployments yet. However it does look very promising.
Interesting blog posts from Plurk who used it for their Comet service:
http://amix.dk/blog/post/19577#Is-node-js-best-for-Comet
http://amix.dk/blog/post/19490
> What strikes me from your comments on thus work is that it seems to be z no-brained. But since you've opted to go with RTC on this I'm wondering what the potential drawback is. Why not just commit it? What is your primary concern?
I have a few concerns:
1. We may simply have an issue with our configuration of DWR or Jetty, and we may be able bump the speed up to a more satisfactory level without major changes
2. Running an additional two services is a pain, and shouldn't be done unless it really is strictly necessary. For example, with the code I've developed you end up running:
- Node.js
- Redis
- Jetty
- Derby
... with relational data stored in Derby and tuples stored in Redis. Its certainly a more complex deployment than we have at present.
3. Integrating participants is less trivial than state data, and would not necessarily have any performance benefit
4. The benefits may be entirely due to websockets vs comet, in which case a websockets implementation in DWR would be the easiest path. However, I haven't seen a lot of discussion of websockets on the DWR lists.
Alternatively, Jetty has WebSockets support as of V7.0.1. However, I'm not aware of any implementation for Tomcat.
> Note I'm not suggesting we commit the code now, so close to a release, but perhaps immediately after the release?
>
> With respect to performance issues, maybe we should make it a priority to do some profiling for the next release?
Agreed, I'd be wary of jumping to conclusions until we've identified where performance issues occur.
> If we do add profiling we could do the node.js in a branch and compare results before merge. That might help us see where poor Wookie performance is being masked by improved node.js performance.
Sounds good to me.
>
> Sent from my mobile device.
>
> On 25 Oct 2010, at 09:07, "Scott Wilson (JIRA)" <ji...@apache.org> wrote:
>
>>
>> [ https://issues.apache.org/jira/browse/WOOKIE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924490#action_12924490 ]
>>
>> Scott Wilson commented on WOOKIE-155:
>> -------------------------------------
>>
>> I built the experimental functionality into a "NodeWave" feature that can be used instead of the existing DWR-based feature and tried it out with some widgets. I'm not surprised that using Node.js+Redis+WebSockets was very, very fast (duh!) but am a bit concerned that DWR+Comet is surprisingly slow.
>>
>> I wonder which part of Wookie might have a performance issue? It could be any one of Comet/DWR, Jetty, the Wookie server code itself, JPA or Derby.
>>
>>
>>> Experimental setup for shared data using websockets, node.js and redis
>>> ----------------------------------------------------------------------
>>>
>>> Key: WOOKIE-155
>>> URL: https://issues.apache.org/jira/browse/WOOKIE-155
>>> Project: Wookie
>>> Issue Type: Improvement
>>> Components: Server
>>> Reporter: Scott Wilson
>>> Priority: Minor
>>> Attachments: chat_dwr_vid.swf, chat_node_vid.swf, wavenoderedis.zip
>>>
>>>
>>> This is an experimental setup that simulates replacing the DWR-based functionality of Wookie's Wave API implementation with one using Node.js, WebSockets, and Redis. The key motivation behind this experiment is to see how much more responsive Wookie shared state widgets can be using a fast Websockets implementation instead of Comet on a typical Java server stack.
>>> To try it out, you need to install Node.js and the SocketIO websockets implementation. You also need to run a Redis server. This file contains the server-side logic.
>>> To run the example:
>>> 1. Start your redis server on the default port using:
>>> ./redis-server
>>> 2. In the folder you unzipped the code into, type:
>>> node server.js
>>> 3. In your browser (Safari and Chrome work well) open each of the testx.html files. Test and Test2 share the same SharedDataKey whereas Test3.html does not. Type in key:value pairs in the text boxes to send deltas to the wave state, and see them updated in other "widget instances".
>>> Note the example is incomplete as it only handles state, not viewer or participants
>>
>> --
>> This message is automatically generated by JIRA.
>> -
>> You can reply to this email to add a comment to the issue online.
>>