You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Trustin Lee <tr...@gmail.com> on 2006/04/26 08:58:34 UTC
Switching codecs in runtime (Was: Removing synchronization on a ProtocolEncoder and a ProtocolDecoder)
On 3/27/06, Niklas Therning <ni...@trillian.se> wrote:
>
> In protocols like SMTP when there are simple line-based commands
> intermixed with raw data (mail data) there are also great opportunities
> for optimization if you write your own codec filter. I've implemented my
> own DecoderFilter which can operate in "data mode". When not in data
> mode the filter will act more or less like an ordinary ProtocolDecoder,
> copying the received buffer to an autoexpanding buffer, decode SMTP
> commands and pass them on to the next filter. When in data mode however
> the filter will simply forward the buffers as they are received without
> any copying (in most cases).
>
> I guess this could be achieved with the MINA codec package but not
> without some tweaking and not as efficiently. Please let me know if I'm
> wrong.
>
> I wouldn't mind adding this filter to MINA if anyone is interested.
It would be nice if we can generalize this behavior. We could then switch
arbitrary set of filters in runtime fairly easily. WDYT?
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: Switching codecs in runtime (Was: Removing synchronization on a ProtocolEncoder and a ProtocolDecoder)
Posted by Trustin Lee <tr...@gmail.com>.
On 6/26/06, Niklas Therning <ni...@trillian.se> wrote:
>
> I've added the flush() method to ProtocolDecoderOutput and
> SimpleProtocolDecoderOutput. I've changed so that a new SPDO is created
> on each call to ProtocolCodecFilter.messageReceived() (it used to be
> cached in the session) since the SPDO have to have a ref to the
> NextFilter and I think it's better not to hold on to an old one. I think
> that's ok since I can't see that messages will ever be left inside the
> queue of the SPDO anyway after message Received() returns. But please
> let me know if I'm wrong and we'll have to think of something else.
Sounds good. Actually creating SPDO every time can resolve possible
concurrency problem we're facing now. Perhaps we need to create SPEO every
time, too?
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: Server closes session with low bandwidth clients
Posted by Trustin Lee <tr...@gmail.com>.
On 6/26/06, Eugene Labunsky <el...@trendmedium.com> wrote:
>
> Hello Trustin,
>
> >All what I can say is
> > that you'd better upgrade to MINA now. Upgrade process is very simple.
> > Please refer to the web site.
>
> I refer web site, but looks like sample for MINA 0.8. I like use last 0.9
> release. Do you have migration sample for 0.9? I can't find in MINA 0.9
> ProtocolProvider Class... Could you advise what I need use?
Could you take a look into the Getting Started section of our on-line web
site? You can browse the SumUp example which demonstratess the use of a
protocol codec.
HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: Server closes session with low bandwidth clients
Posted by Eugene Labunsky <el...@trendmedium.com>.
Hello Trustin,
>All what I can say is
> that you'd better upgrade to MINA now. Upgrade process is very simple.
> Please refer to the web site.
I refer web site, but looks like sample for MINA 0.8. I like use last 0.9
release. Do you have migration sample for 0.9? I can't find in MINA 0.9
ProtocolProvider Class... Could you advise what I need use?
Regards,
Eugene Labunsky.
http://www.TrendMedium.com
Re: Server closes session with low bandwidth clients
Posted by Trustin Lee <tr...@gmail.com>.
Hello Eugene,
On 6/26/06, Eugene Labunsky <el...@trendmedium.com> wrote:
>
> Hello Trustin,
>
> Could you help me with my small problems?
> I have developed server with Netty tl-netty2-1.9.2. This server send a lot
> of data to client each time (3-5kb per request) with a lot of reply lines.
> Each time I'm using something like this:
>
> ReplyMessage rm = new ReplyMessage("SomeReplyString");
> session.write(rm);
>
> Server working fine with high bandwidth clients, but stopped offen sending
> data to low bandwidth clients... Could you advise anything?
Unfortunately, this mailing list for MINA, not Netty. All what I can say is
that you'd better upgrade to MINA now. Upgrade process is very simple.
Please refer to the web site.
HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Server closes session with low bandwidth clients
Posted by Eugene Labunsky <el...@trendmedium.com>.
Hello Trustin,
Could you help me with my small problems?
I have developed server with Netty tl-netty2-1.9.2. This server send a lot
of data to client each time (3-5kb per request) with a lot of reply lines.
Each time I'm using something like this:
ReplyMessage rm = new ReplyMessage("SomeReplyString");
session.write(rm);
Server working fine with high bandwidth clients, but stopped offen sending
data to low bandwidth clients... Could you advise anything?
Thank you for your great work!
Regards,
Eugene Labunsky.
http://www.TrendMedium.com
Re: Switching codecs in runtime (Was: Removing synchronization on
a ProtocolEncoder and a ProtocolDecoder)
Posted by Niklas Therning <ni...@trillian.se>.
Niklas Therning wrote:
> Trustin Lee wrote:
>> Ahh didn't we have a flush() method in ProtocolDecoderOutput? Let's add it
>> then! :)
>
> Ok, great! :) I'll have a look at it.
>
>
I've added the flush() method to ProtocolDecoderOutput and
SimpleProtocolDecoderOutput. I've changed so that a new SPDO is created
on each call to ProtocolCodecFilter.messageReceived() (it used to be
cached in the session) since the SPDO have to have a ref to the
NextFilter and I think it's better not to hold on to an old one. I think
that's ok since I can't see that messages will ever be left inside the
queue of the SPDO anyway after message Received() returns. But please
let me know if I'm wrong and we'll have to think of something else.
--
Niklas Therning
Software Architect
www.spamdrain.net
Re: Switching codecs in runtime (Was: Removing synchronization on
a ProtocolEncoder and a ProtocolDecoder)
Posted by Niklas Therning <ni...@trillian.se>.
Trustin Lee wrote:
>
> Ahh didn't we have a flush() method in ProtocolDecoderOutput? Let's add it
> then! :)
Ok, great! :) I'll have a look at it.
--
Niklas Therning
Software Architect
www.spamdrain.net
Re: Switching codecs in runtime (Was: Removing synchronization on a ProtocolEncoder and a ProtocolDecoder)
Posted by Trustin Lee <tr...@gmail.com>.
On 6/10/06, Niklas Therning <ni...@trillian.se> wrote:
>
> Trustin Lee wrote:
> > The separation between IoHandler and protocol codec loses its meaning if
> > IoHandler takes care of the codec's state. That's why I don't want
> > IoHandler affect any state of decoder or encoder. And as I mentioned,
> the
> > state of the decoder might not change immediately in case that there's a
> > thread pool filter between the protocolcodecfilter and IoHandler.
>
> When the client sends the DATA command I do the following in my IoHandler:
>
> session.setAttribute(CommandProtocolDecoder.DATA_MODE);
> session.write(new OkResponse());
>
> From here on my CommandProtocolDecoder will forward the received
> ByteBuffers without doing any special processing. Now consider what
> would happen if the IoHandler responds with a negative response instead.
> In that case my CommandProtocolDecoder shouldn't switch to data mode but
> should continue to decode as usual. If I'm not allowed to control this
> from my IoHandler my decoder and encoder would both have to be aware of
> (at least partially) where in the protocol flow the current session is.
> Of course my IoHandler also needs to now this so instead of my IoHandler
> being the only one with any knowledge of where we are in the flow all of
> them have to keep track of what has happened previously in order to know
> what to do next.
You are correct. Sorry for stating a wrong point.
> My suggestion is still to provide a switchable encoder/decoder.
> > TokenSelectorStream in ANTLR shows a nice example regarding this
> pattern:
> >
> > http://www.antlr.org/doc/streams.html#lexerstates
> >
> > For example:
> >
> > SwitchableProtocolDecoder d = new SwitchableProtocolDecoder();
> > d.register("default", new DefaultProtocolDecoder());
> > d.register("attachment", new AttachmentDecoder());
> > d.push("default");
> >
> > ...
> > if (timeToDecodeAttachment) d.push("attachment");
> > ...
> > if (finishedDecodingAttachment) d.pop();
> > ...
>
> I think this is a very nice idea. Would my IoHandler control the pushes
> and pops? In that case you would still need the flush() method in PDO,
> right? Otherwise you cannot switch decoder in between two messages if
> they arrive in the same buffer. And the issue with using a
> ThreadPoolFilter between the ProtocolCodecFilter and IoHandler would
> still remain. Or am I missing something?
You're right. This selector pattern is not necessarilly related with the
issue you raised. It is just a good-to-have class.
Please remember that the only change to MINA I'm suggesting is to
> introduce the flush() method in PDO so that I could control flushes from
> within my decoder. The PDO doesn't even have to be pluggable.
Ahh didn't we have a flush() method in ProtocolDecoderOutput? Let's add it
then! :)
Thanks,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: Switching codecs in runtime (Was: Removing synchronization on
a ProtocolEncoder and a ProtocolDecoder)
Posted by Niklas Therning <ni...@trillian.se>.
Trustin Lee wrote:
> The separation between IoHandler and protocol codec loses its meaning if
> IoHandler takes care of the codec's state. That's why I don't want
> IoHandler affect any state of decoder or encoder. And as I mentioned, the
> state of the decoder might not change immediately in case that there's a
> thread pool filter between the protocolcodecfilter and IoHandler.
When the client sends the DATA command I do the following in my IoHandler:
session.setAttribute(CommandProtocolDecoder.DATA_MODE);
session.write(new OkResponse());
>From here on my CommandProtocolDecoder will forward the received
ByteBuffers without doing any special processing. Now consider what
would happen if the IoHandler responds with a negative response instead.
In that case my CommandProtocolDecoder shouldn't switch to data mode but
should continue to decode as usual. If I'm not allowed to control this
from my IoHandler my decoder and encoder would both have to be aware of
(at least partially) where in the protocol flow the current session is.
Of course my IoHandler also needs to now this so instead of my IoHandler
being the only one with any knowledge of where we are in the flow all of
them have to keep track of what has happened previously in order to know
what to do next.
> My suggestion is still to provide a switchable encoder/decoder.
> TokenSelectorStream in ANTLR shows a nice example regarding this pattern:
>
> http://www.antlr.org/doc/streams.html#lexerstates
>
> For example:
>
> SwitchableProtocolDecoder d = new SwitchableProtocolDecoder();
> d.register("default", new DefaultProtocolDecoder());
> d.register("attachment", new AttachmentDecoder());
> d.push("default");
>
> ...
> if (timeToDecodeAttachment) d.push("attachment");
> ...
> if (finishedDecodingAttachment) d.pop();
> ...
I think this is a very nice idea. Would my IoHandler control the pushes
and pops? In that case you would still need the flush() method in PDO,
right? Otherwise you cannot switch decoder in between two messages if
they arrive in the same buffer. And the issue with using a
ThreadPoolFilter between the ProtocolCodecFilter and IoHandler would
still remain. Or am I missing something?
Please remember that the only change to MINA I'm suggesting is to
introduce the flush() method in PDO so that I could control flushes from
within my decoder. The PDO doesn't even have to be pluggable.
--
Niklas Therning
Software Architect
www.spamdrain.net
Re: Switching codecs in runtime (Was: Removing synchronization on a ProtocolEncoder and a ProtocolDecoder)
Posted by Trustin Lee <tr...@gmail.com>.
The separation between IoHandler and protocol codec loses its meaning if
IoHandler takes care of the codec's state. That's why I don't want
IoHandler affect any state of decoder or encoder. And as I mentioned, the
state of the decoder might not change immediately in case that there's a
thread pool filter between the protocolcodecfilter and IoHandler.
My suggestion is still to provide a switchable encoder/decoder.
TokenSelectorStream in ANTLR shows a nice example regarding this pattern:
http://www.antlr.org/doc/streams.html#lexerstates
For example:
SwitchableProtocolDecoder d = new SwitchableProtocolDecoder();
d.register("default", new DefaultProtocolDecoder());
d.register("attachment", new AttachmentDecoder());
d.push("default");
...
if (timeToDecodeAttachment) d.push("attachment");
...
if (finishedDecodingAttachment) d.pop();
...
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: Switching codecs in runtime (Was: Removing synchronization on
a ProtocolEncoder and a ProtocolDecoder)
Posted by Niklas Therning <ni...@trillian.se>.
Niklas Therning wrote:
> Trustin Lee wrote:
>> Hi Niklas,
>>
>> On 4/26/06, Niklas Therning <ni...@trillian.se> wrote:
>>> Trustin Lee wrote:
>>>> On 3/27/06, Niklas Therning <ni...@trillian.se> wrote:
>>>>
>>>>> In protocols like SMTP when there are simple line-based commands
>>>>> intermixed with raw data (mail data) there are also great opportunities
>>>>> for optimization if you write your own codec filter. I've
>>> implemented my
>>>>> own DecoderFilter which can operate in "data mode". When not in data
>>>>> mode the filter will act more or less like an ordinary ProtocolDecoder,
>>>>> copying the received buffer to an autoexpanding buffer, decode SMTP
>>>>> commands and pass them on to the next filter. When in data mode however
>>>>> the filter will simply forward the buffers as they are received without
>>>>> any copying (in most cases).
>>>>>
>>>>> I guess this could be achieved with the MINA codec package but not
>>>>> without some tweaking and not as efficiently. Please let me know if I'm
>>>>> wrong.
>>>>>
>>>>> I wouldn't mind adding this filter to MINA if anyone is interested.
>>>>
>>>>
>>>> It would be nice if we can generalize this behavior. We could then
>>> switch
>>>> arbitrary set of filters in runtime fairly easily. WDYT?
>>>>
>>>> Trustin
>>> I think DIRMINA-201 solves the efficiency issue I was referring to. The
>>> only thing that would have to change to make ProtocolCodecFilter suit my
>>> needs is that SimpleProtocolDecoderOut shouldn't queue messages but
>>> rather forward them to the nextFilter right away as they are written.
>>> Then my IoHandler would be able to instruct my Decoder instance to
>>> change its state and decode differently depending on what the previous
>>> message was.
>>
>> If we directly call the IoHandler.messageReceived() when we call
>> ProtocolDecoderOut.write(), any exceptions raised by the handler can
>> interrupt the decoding process. SimpleProtocolDecoderOut.write() could
>> catch the exception but I think it cannot fire exceptionCaught event
>> correctly because it cannot access AbstractIoFilterChain without breaking
>> the OO design. Moreover, we cannot guarentee that
>> IoHandler.messageReceived()
>> is executed in the same thread with ProtocolDecoderOut.write() because
>> there can be an extra thread pool between the codec and the handler.
>>
>> Is it a problem for you to change the decoder state in the decoder itself?
>
> I'd rather keep my decoder/encoder oblivious of state. I'd like to keep
> them as simple as possible.
>
> Here's what I propose: make the ProtocolDecoderOut used by
> ProtocolCodecFilter pluggable and make the default behave exactly as
> today. Add the method flush() to ProtocolDecoderOut. Change
> ProtocolCodecFilter.messageReceived() slightly:
>
> ProtocolDecoderOutput decoderOut = getDecoderOut(nextFilter,session);
>
> try
> {
> decoder.decode( session, in, decoderOut );
> }
> catch( Throwable t )
> {
> ... // No change here
> }
> finally
> {
> // Dispose the decoder if this session is connectionless.
> ...
>
> // Release the read buffer.
> in.release();
>
> decoderOut.flush();
> }
>
> The code
> Queue queue = decoderOut.getMessageQueue();
> while( !queue.isEmpty() )
> {
> nextFilter.messageReceived( session, queue.pop() );
> }
> would move into the flush() method of SimpleProtocolDecoderOutput.
>
> Now I could extend ProtocolCodecFilter and override getDecoderOut() to
> use my non-queuing PDO instead. It would look something like:
>
> public class NonQueingProtocolDecoderOut implements ProtocolDecoderOutput
> {
> private final IoSession session;
> private final NextFilter nextFilter;
> public NonQueingProtocolDecoderOut(IoSession session, NextFilter
> nextFilter)
> {
> this.session = session;
> this.netxFilter = nextFilter;
> }
>
> public void write( Object message )
> {
> nextFilter.messageReceived( session, message );
> }
>
> public void flush() {
> }
> }
>
> This solution avoids calling IoHandler.messageReceived() directly. It
> uses the NextFilter which will handle the exceptionCaught() as expected
> so the exception will never be thrown into the PDO.
>
> The threading issue is of course still a problem but that is something
> we would need to document. It doesn't make any sense to add a
> ThredPoolFilter after the ProtocolCodecFilter if you want to achieve
> what I want to.
>
> I guess the flush() method should only be called by MINA so adding that
> to the interface could be a problem. But it could also be a feature? I
> could actually achieve what I want by calling flush from my decoder with
> the SimpleProtocolDecoderOutput!
>
> WDYT?
>
If no one objects I will go ahead and make these changes.
One concern I have is with the NextFilter. Right now the PDO is being
cached in the session. Using the approach I described above it will hold
on to the NextFilter instance. Is that safe? Is there any reason why we
can't create a new PDO each time we need one instead of caching one in
the session?
--
Niklas Therning
Software Architect
www.spamdrain.net
Re: Switching codecs in runtime (Was: Removing synchronization on
a ProtocolEncoder and a ProtocolDecoder)
Posted by Niklas Therning <ni...@trillian.se>.
Trustin Lee wrote:
> Hi Niklas,
>
> On 4/26/06, Niklas Therning <ni...@trillian.se> wrote:
>>
>> Trustin Lee wrote:
>> > On 3/27/06, Niklas Therning <ni...@trillian.se> wrote:
>> >
>> >>In protocols like SMTP when there are simple line-based commands
>> >>intermixed with raw data (mail data) there are also great opportunities
>> >>for optimization if you write your own codec filter. I've
>> implemented my
>> >>own DecoderFilter which can operate in "data mode". When not in data
>> >>mode the filter will act more or less like an ordinary ProtocolDecoder,
>> >>copying the received buffer to an autoexpanding buffer, decode SMTP
>> >>commands and pass them on to the next filter. When in data mode however
>> >>the filter will simply forward the buffers as they are received without
>> >>any copying (in most cases).
>> >>
>> >>I guess this could be achieved with the MINA codec package but not
>> >>without some tweaking and not as efficiently. Please let me know if I'm
>> >>wrong.
>> >>
>> >>I wouldn't mind adding this filter to MINA if anyone is interested.
>> >
>> >
>> >
>> > It would be nice if we can generalize this behavior. We could then
>> switch
>> > arbitrary set of filters in runtime fairly easily. WDYT?
>> >
>> > Trustin
>>
>> I think DIRMINA-201 solves the efficiency issue I was referring to. The
>> only thing that would have to change to make ProtocolCodecFilter suit my
>> needs is that SimpleProtocolDecoderOut shouldn't queue messages but
>> rather forward them to the nextFilter right away as they are written.
>> Then my IoHandler would be able to instruct my Decoder instance to
>> change its state and decode differently depending on what the previous
>> message was.
>
>
> If we directly call the IoHandler.messageReceived() when we call
> ProtocolDecoderOut.write(), any exceptions raised by the handler can
> interrupt the decoding process. SimpleProtocolDecoderOut.write() could
> catch the exception but I think it cannot fire exceptionCaught event
> correctly because it cannot access AbstractIoFilterChain without breaking
> the OO design. Moreover, we cannot guarentee that
> IoHandler.messageReceived()
> is executed in the same thread with ProtocolDecoderOut.write() because
> there can be an extra thread pool between the codec and the handler.
>
> Is it a problem for you to change the decoder state in the decoder itself?
I'd rather keep my decoder/encoder oblivious of state. I'd like to keep
them as simple as possible.
Here's what I propose: make the ProtocolDecoderOut used by
ProtocolCodecFilter pluggable and make the default behave exactly as
today. Add the method flush() to ProtocolDecoderOut. Change
ProtocolCodecFilter.messageReceived() slightly:
ProtocolDecoderOutput decoderOut = getDecoderOut(nextFilter,session);
try
{
decoder.decode( session, in, decoderOut );
}
catch( Throwable t )
{
... // No change here
}
finally
{
// Dispose the decoder if this session is connectionless.
...
// Release the read buffer.
in.release();
decoderOut.flush();
}
The code
Queue queue = decoderOut.getMessageQueue();
while( !queue.isEmpty() )
{
nextFilter.messageReceived( session, queue.pop() );
}
would move into the flush() method of SimpleProtocolDecoderOutput.
Now I could extend ProtocolCodecFilter and override getDecoderOut() to
use my non-queuing PDO instead. It would look something like:
public class NonQueingProtocolDecoderOut implements ProtocolDecoderOutput
{
private final IoSession session;
private final NextFilter nextFilter;
public NonQueingProtocolDecoderOut(IoSession session, NextFilter
nextFilter)
{
this.session = session;
this.netxFilter = nextFilter;
}
public void write( Object message )
{
nextFilter.messageReceived( session, message );
}
public void flush() {
}
}
This solution avoids calling IoHandler.messageReceived() directly. It
uses the NextFilter which will handle the exceptionCaught() as expected
so the exception will never be thrown into the PDO.
The threading issue is of course still a problem but that is something
we would need to document. It doesn't make any sense to add a
ThredPoolFilter after the ProtocolCodecFilter if you want to achieve
what I want to.
I guess the flush() method should only be called by MINA so adding that
to the interface could be a problem. But it could also be a feature? I
could actually achieve what I want by calling flush from my decoder with
the SimpleProtocolDecoderOutput!
WDYT?
--
Niklas Therning
Software Architect
www.spamdrain.net
Re: Switching codecs in runtime (Was: Removing synchronization on a ProtocolEncoder and a ProtocolDecoder)
Posted by Trustin Lee <tr...@gmail.com>.
Hi Niklas,
On 4/26/06, Niklas Therning <ni...@trillian.se> wrote:
>
> Trustin Lee wrote:
> > On 3/27/06, Niklas Therning <ni...@trillian.se> wrote:
> >
> >>In protocols like SMTP when there are simple line-based commands
> >>intermixed with raw data (mail data) there are also great opportunities
> >>for optimization if you write your own codec filter. I've implemented my
> >>own DecoderFilter which can operate in "data mode". When not in data
> >>mode the filter will act more or less like an ordinary ProtocolDecoder,
> >>copying the received buffer to an autoexpanding buffer, decode SMTP
> >>commands and pass them on to the next filter. When in data mode however
> >>the filter will simply forward the buffers as they are received without
> >>any copying (in most cases).
> >>
> >>I guess this could be achieved with the MINA codec package but not
> >>without some tweaking and not as efficiently. Please let me know if I'm
> >>wrong.
> >>
> >>I wouldn't mind adding this filter to MINA if anyone is interested.
> >
> >
> >
> > It would be nice if we can generalize this behavior. We could then
> switch
> > arbitrary set of filters in runtime fairly easily. WDYT?
> >
> > Trustin
>
> I think DIRMINA-201 solves the efficiency issue I was referring to. The
> only thing that would have to change to make ProtocolCodecFilter suit my
> needs is that SimpleProtocolDecoderOut shouldn't queue messages but
> rather forward them to the nextFilter right away as they are written.
> Then my IoHandler would be able to instruct my Decoder instance to
> change its state and decode differently depending on what the previous
> message was.
If we directly call the IoHandler.messageReceived() when we call
ProtocolDecoderOut.write(), any exceptions raised by the handler can
interrupt the decoding process. SimpleProtocolDecoderOut.write() could
catch the exception but I think it cannot fire exceptionCaught event
correctly because it cannot access AbstractIoFilterChain without breaking
the OO design. Moreover, we cannot guarentee that IoHandler.messageReceived()
is executed in the same thread with ProtocolDecoderOut.write() because
there can be an extra thread pool between the codec and the handler.
Is it a problem for you to change the decoder state in the decoder itself?
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: IoSession writes
Posted by Trustin Lee <tr...@gmail.com>.
On 4/27/06, Samuel Doyle <sd...@yahoo.com> wrote:
>
> I'm a bit unclear on the javadoc description of this
> method. The following is the actual description.
>
> WriteFuture write(Object message)
>
> Writes the specified message to remote peer. This
> operation is asynchronous;
> IoHandler.messageSent(IoSession, Object) will be
> invoked when the message is actually sent to remote
> peer. You can also ait for the returned WriteFuture if
> you want to wait for the session actually closed.
>
> The last part is where it is a bit unclear to me. Is
> it saying that when you join on the future the session
> is to be closed after that? Or does it mean that the
> actual write operation has fully completed and the
> session remains open?
It's an error in the documentation. Thanks for the notification!
session.write(...).join() just wait for the message to be written.
In addition, is there a synchronized version of this
> method? I'm just curious what is the best way to
> handle a case where you may have multiple threads that
> want to send packets of data to the same session.
> Basically you have multiple client sessions driving
> messages down the same piple to the server, the data
> can not be mixed. Should I do my own locking in this
> case?
Just add .join() after your write() call. It never returns null. Or do we
need a synchronous method? I didn't add it because .join() is simple
enough. Perhaps I need to add this idiom to the documentation, too. :)
HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: Attempt to upgrade to 0.9.3
Posted by Trustin Lee <tr...@gmail.com>.
The main advantage of ServiceRegistry was in that it automatically adds
essential filters such as ThreadPoolFilter. We added ThreadModel property
in IoServiceConfig so you can configure your favorite thread model very
easily now, and thus we decided to remove ServiceRegistry.
The default thread model is a very popular thread model with one thread pool
filter. So you don't need to add your thread pool filter manually at all
since 0.9.3.
Please let us know if there're features you miss from ServiceRegistry
interface.
HTH,
Trustin
On 4/27/06, Samuel Doyle <sd...@yahoo.com> wrote:
>
> So I attempted to upgrade my 0.9.0 prototype to 0.9.3
> I was hoping replacing the jar with the new core jar
> would do the trick, I guess that was wishful thinking.
> :)
> There was a fair bit of code involved in this so any
> recommendations on the best way to tackle getting this
> working again with the latest version would be
> appreciated.
>
> Thanks, S.D.
>
> compile:
> [javac] Compiling 21 source files to
> lib/java/classes
> [javac] BaseProcessor.java:23: package
> org.apache.mina.registry does not exist
> [javac] import org.apache.mina.registry.Service;
> [javac] ^
> [javac] BaseProcessor.java:24: package
> org.apache.mina.registry does not exist
> [javac] import
> org.apache.mina.registry.ServiceRegistry;
> [javac] ^
> [javac] BaseProcessor.java:25: package
> org.apache.mina.registry does not exist
> [javac] import
> org.apache.mina.registry.SimpleServiceRegistry;
> [javac] ^
> [javac] BaseProcessor.java:38: cannot find symbol
> [javac] symbol : class ServiceRegistry
> [javac] location: class
> com.wm.finance.epay.BaseProcessor
> [javac] protected ServiceRegistry myRegistry =
> null;
> [javac] ^
> [javac] BaseProcessor.java:109: cannot find symbol
> [javac] symbol : class ServiceRegistry
> [javac] location: class
> com.wm.finance.epay.BaseProcessor
> [javac] protected abstract void addLogger(
> ServiceRegistry registry ) throws Exception;
> [javac] ^
> [javac] ShoppingCardProcessor.java:32: package
> org.apache.mina.registry does not exist
> [javac] import
> org.apache.mina.registry.ServiceRegistry;
> [javac] ^
> [javac] RTCCSessionHandler.java:34: cannot find
> symbol
> [javac] symbol : class SocketSession
> [javac] location: package
> org.apache.mina.transport.socket.nio
> [javac] import
> org.apache.mina.transport.socket.nio.SocketSession;
> [javac]
> ^
> [javac] EPAYSessionHandler.java:36: cannot find
> symbol
> [javac] symbol : class SocketSession
> [javac] location: package
> org.apache.mina.transport.socket.nio
> [javac] import
> org.apache.mina.transport.socket.nio.SocketSession;
> [javac]
> ^
> [javac] ShoppingCardProcessor.java:105: cannot
> find symbol
> [javac] symbol : class ServiceRegistry
> [javac] location: class
> com.wm.finance.epay.shoppingcard.ShoppingCardProcessor
> [javac] protected void addLogger(
> ServiceRegistry registry ) throws Exception
> [javac] ^
> [javac] BaseProcessor.java:67: cannot find symbol
> [javac] symbol : class SimpleServiceRegistry
> [javac] location: class
> com.wm.finance.epay.BaseProcessor
> [javac] this.myRegistry = new
> SimpleServiceRegistry();
> [javac] ^
> [javac] BaseProcessor.java:83: cannot find symbol
> [javac] symbol : class Service
> [javac] location: class
> com.wm.finance.epay.BaseProcessor
> [javac] Service service = new Service(
> this.myName, TransportType.SOCKET, this.port );
> [javac] ^
> [javac] BaseProcessor.java:83: cannot find symbol
> [javac] symbol : class Service
> [javac] location: class
> com.wm.finance.epay.BaseProcessor
> [javac] Service service = new Service(
> this.myName, TransportType.SOCKET, this.port );
> [javac] ^
> [javac] RTCCSessionHandler.java:76: cannot find
> symbol
> [javac] symbol : class SocketSession
> [javac] location: class
> com.wm.finance.epay.shoppingcard.comms.RTCCSessionHandler
> [javac] if( session instanceof SocketSession
> )
> [javac] ^
> [javac] RTCCSessionHandler.java:78: cannot find
> symbol
> [javac] symbol : class SocketSession
> [javac] location: class
> com.wm.finance.epay.shoppingcard.comms.RTCCSessionHandler
> [javac] ( ( SocketSession ) session
> ).setSessionReceiveBufferSize( 2048 );
> [javac] ^
> [javac] EPAYSessionHandler.java:78: cannot find
> symbol
> [javac] symbol : class SocketSession
> [javac] location: class
> com.wm.finance.epay.shoppingcard.comms.EPAYSessionHandler
> [javac] if( session instanceof SocketSession
> )
> [javac] ^
> [javac] EPAYSessionHandler.java:80: cannot find
> symbol
> [javac] symbol : class SocketSession
> [javac] location: class
> com.wm.finance.epay.shoppingcard.comms.EPAYSessionHandler
> [javac] ( ( SocketSession ) session
> ).setSessionReceiveBufferSize( 2048 );
> [javac] ^
> [javac] EPAYSessionPool.java:84: cannot find
> symbol
> [javac] symbol : method setConnectTimeout(int)
> [javac] location: interface
> org.apache.mina.common.IoConnector
> [javac] this.connector.setConnectTimeout(
> this.connectTimeOut );
> [javac] ^
> [javac] 17 errors
>
>
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: What happened to SocketConnector.setConnectTimeout ?
Posted by Trustin Lee <tr...@gmail.com>.
On 4/27/06, Samuel Doyle <sd...@yahoo.com> wrote:
>
> Alright I assume this is how you do it. Can someone
> confirm?
>
> IoConnector connector = new SocketConnector()
> BaseIoConnectorConfig connectorConfig = new
> SocketConnectorConfig();
>
> connectorConfig.setConnectTimeout( connectTimeOut );
> connector.connect( host, sessionHandler,
> connectorConfig );
BaseIoConnectorConfig is an internal support class. Please don't use it.
IoConnectorConfig connectorConfig = new SocketConnectorConfig();
will suffice.
Otherwise, if you want to just override the default configuration for the
connector you created:
connector,getDefaultConfig().setConnectTimeout( connectTimeout );
HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: What happened to SocketConnector.setConnectTimeout ?
Posted by Samuel Doyle <sd...@yahoo.com>.
Hi,
Yes I do something similar to handle legacy client -->
mina connections. The socketconnecor is required to go
from mina --> legacy server, well at least that is
what I assume based on looking at examples and apis.
I'm using mina to act as a proxy server.
S.D.
--- Enrique Rodriguez <en...@gmail.com> wrote:
> Samuel Doyle wrote:
> > Alright I assume this is how you do it. Can
> someone
> > confirm?
> >
> > IoConnector connector = new SocketConnector()
> > BaseIoConnectorConfig connectorConfig = new
> > SocketConnectorConfig();
> >
> > connectorConfig.setConnectTimeout( connectTimeOut
> );
> > connector.connect( host, sessionHandler,
> > connectorConfig );
>
> I've been doing:
>
> private IoAcceptor acceptor;
> private IoHandler handler;
>
> handler = new KerberosProtocolHandler();
> acceptor.bind( new InetSocketAddress( port ),
> handler );
>
> Enrique
>
Re: What happened to SocketConnector.setConnectTimeout ?
Posted by Enrique Rodriguez <en...@gmail.com>.
Samuel Doyle wrote:
> Alright I assume this is how you do it. Can someone
> confirm?
>
> IoConnector connector = new SocketConnector()
> BaseIoConnectorConfig connectorConfig = new
> SocketConnectorConfig();
>
> connectorConfig.setConnectTimeout( connectTimeOut );
> connector.connect( host, sessionHandler,
> connectorConfig );
I've been doing:
private IoAcceptor acceptor;
private IoHandler handler;
handler = new KerberosProtocolHandler();
acceptor.bind( new InetSocketAddress( port ), handler );
Enrique
Re: What happened to SocketConnector.setConnectTimeout ?
Posted by Samuel Doyle <sd...@yahoo.com>.
Alright I assume this is how you do it. Can someone
confirm?
IoConnector connector = new SocketConnector()
BaseIoConnectorConfig connectorConfig = new
SocketConnectorConfig();
connectorConfig.setConnectTimeout( connectTimeOut );
connector.connect( host, sessionHandler,
connectorConfig );
Thanks, S.D.
--- Samuel Doyle <sd...@yahoo.com> wrote:
> It appears that the SocketConnector parent class
> org.apache.mina.common.support.DelegatedIoConnector
> is
> missing from the javadocs.
>
> S.D.
>
> --- Samuel Doyle <sd...@yahoo.com> wrote:
>
> > Where did this go? What should I use to replace
> it?
> >
> > Thanks, S.D.
> >
>
>
Re: What happened to SocketConnector.setConnectTimeout ?
Posted by Samuel Doyle <sd...@yahoo.com>.
It appears that the SocketConnector parent class
org.apache.mina.common.support.DelegatedIoConnector is
missing from the javadocs.
S.D.
--- Samuel Doyle <sd...@yahoo.com> wrote:
> Where did this go? What should I use to replace it?
>
> Thanks, S.D.
>
What happened to SocketConnector.setConnectTimeout ?
Posted by Samuel Doyle <sd...@yahoo.com>.
Where did this go? What should I use to replace it?
Thanks, S.D.
Re: Attempt to upgrade to 0.9.3
Posted by peter royal <pr...@apache.org>.
On Apr 26, 2006, at 7:18 PM, Samuel Doyle wrote:
> There was a fair bit of code involved in this so any
> recommendations on the best way to tackle getting this
> working again with the latest version would be
> appreciated.
Take a look at the examples.. Service and ServiceRegistry have been
removed.
-pete
--
proyal@apache.org - http://fotap.org/~osi
Attempt to upgrade to 0.9.3
Posted by Samuel Doyle <sd...@yahoo.com>.
So I attempted to upgrade my 0.9.0 prototype to 0.9.3
I was hoping replacing the jar with the new core jar
would do the trick, I guess that was wishful thinking.
:)
There was a fair bit of code involved in this so any
recommendations on the best way to tackle getting this
working again with the latest version would be
appreciated.
Thanks, S.D.
compile:
[javac] Compiling 21 source files to
lib/java/classes
[javac] BaseProcessor.java:23: package
org.apache.mina.registry does not exist
[javac] import org.apache.mina.registry.Service;
[javac] ^
[javac] BaseProcessor.java:24: package
org.apache.mina.registry does not exist
[javac] import
org.apache.mina.registry.ServiceRegistry;
[javac] ^
[javac] BaseProcessor.java:25: package
org.apache.mina.registry does not exist
[javac] import
org.apache.mina.registry.SimpleServiceRegistry;
[javac] ^
[javac] BaseProcessor.java:38: cannot find symbol
[javac] symbol : class ServiceRegistry
[javac] location: class
com.wm.finance.epay.BaseProcessor
[javac] protected ServiceRegistry myRegistry =
null;
[javac] ^
[javac] BaseProcessor.java:109: cannot find symbol
[javac] symbol : class ServiceRegistry
[javac] location: class
com.wm.finance.epay.BaseProcessor
[javac] protected abstract void addLogger(
ServiceRegistry registry ) throws Exception;
[javac] ^
[javac] ShoppingCardProcessor.java:32: package
org.apache.mina.registry does not exist
[javac] import
org.apache.mina.registry.ServiceRegistry;
[javac] ^
[javac] RTCCSessionHandler.java:34: cannot find
symbol
[javac] symbol : class SocketSession
[javac] location: package
org.apache.mina.transport.socket.nio
[javac] import
org.apache.mina.transport.socket.nio.SocketSession;
[javac]
^
[javac] EPAYSessionHandler.java:36: cannot find
symbol
[javac] symbol : class SocketSession
[javac] location: package
org.apache.mina.transport.socket.nio
[javac] import
org.apache.mina.transport.socket.nio.SocketSession;
[javac]
^
[javac] ShoppingCardProcessor.java:105: cannot
find symbol
[javac] symbol : class ServiceRegistry
[javac] location: class
com.wm.finance.epay.shoppingcard.ShoppingCardProcessor
[javac] protected void addLogger(
ServiceRegistry registry ) throws Exception
[javac] ^
[javac] BaseProcessor.java:67: cannot find symbol
[javac] symbol : class SimpleServiceRegistry
[javac] location: class
com.wm.finance.epay.BaseProcessor
[javac] this.myRegistry = new
SimpleServiceRegistry();
[javac] ^
[javac] BaseProcessor.java:83: cannot find symbol
[javac] symbol : class Service
[javac] location: class
com.wm.finance.epay.BaseProcessor
[javac] Service service = new Service(
this.myName, TransportType.SOCKET, this.port );
[javac] ^
[javac] BaseProcessor.java:83: cannot find symbol
[javac] symbol : class Service
[javac] location: class
com.wm.finance.epay.BaseProcessor
[javac] Service service = new Service(
this.myName, TransportType.SOCKET, this.port );
[javac] ^
[javac] RTCCSessionHandler.java:76: cannot find
symbol
[javac] symbol : class SocketSession
[javac] location: class
com.wm.finance.epay.shoppingcard.comms.RTCCSessionHandler
[javac] if( session instanceof SocketSession
)
[javac] ^
[javac] RTCCSessionHandler.java:78: cannot find
symbol
[javac] symbol : class SocketSession
[javac] location: class
com.wm.finance.epay.shoppingcard.comms.RTCCSessionHandler
[javac] ( ( SocketSession ) session
).setSessionReceiveBufferSize( 2048 );
[javac] ^
[javac] EPAYSessionHandler.java:78: cannot find
symbol
[javac] symbol : class SocketSession
[javac] location: class
com.wm.finance.epay.shoppingcard.comms.EPAYSessionHandler
[javac] if( session instanceof SocketSession
)
[javac] ^
[javac] EPAYSessionHandler.java:80: cannot find
symbol
[javac] symbol : class SocketSession
[javac] location: class
com.wm.finance.epay.shoppingcard.comms.EPAYSessionHandler
[javac] ( ( SocketSession ) session
).setSessionReceiveBufferSize( 2048 );
[javac] ^
[javac] EPAYSessionPool.java:84: cannot find
symbol
[javac] symbol : method setConnectTimeout(int)
[javac] location: interface
org.apache.mina.common.IoConnector
[javac] this.connector.setConnectTimeout(
this.connectTimeOut );
[javac] ^
[javac] 17 errors
IoSession writes
Posted by Samuel Doyle <sd...@yahoo.com>.
I'm a bit unclear on the javadoc description of this
method. The following is the actual description.
WriteFuture write(Object message)
Writes the specified message to remote peer. This
operation is asynchronous;
IoHandler.messageSent(IoSession, Object) will be
invoked when the message is actually sent to remote
peer. You can also ait for the returned WriteFuture if
you want to wait for the session actually closed.
The last part is where it is a bit unclear to me. Is
it saying that when you join on the future the session
is to be closed after that? Or does it mean that the
actual write operation has fully completed and the
session remains open?
In addition, is there a synchronized version of this
method? I'm just curious what is the best way to
handle a case where you may have multiple threads that
want to send packets of data to the same session.
Basically you have multiple client sessions driving
messages down the same piple to the server, the data
can not be mixed. Should I do my own locking in this
case?
Thanks, S.D.
Re: Switching codecs in runtime (Was: Removing synchronization on
a ProtocolEncoder and a ProtocolDecoder)
Posted by Niklas Therning <ni...@trillian.se>.
Trustin Lee wrote:
> On 3/27/06, Niklas Therning <ni...@trillian.se> wrote:
>
>>In protocols like SMTP when there are simple line-based commands
>>intermixed with raw data (mail data) there are also great opportunities
>>for optimization if you write your own codec filter. I've implemented my
>>own DecoderFilter which can operate in "data mode". When not in data
>>mode the filter will act more or less like an ordinary ProtocolDecoder,
>>copying the received buffer to an autoexpanding buffer, decode SMTP
>>commands and pass them on to the next filter. When in data mode however
>>the filter will simply forward the buffers as they are received without
>>any copying (in most cases).
>>
>>I guess this could be achieved with the MINA codec package but not
>>without some tweaking and not as efficiently. Please let me know if I'm
>>wrong.
>>
>>I wouldn't mind adding this filter to MINA if anyone is interested.
>
>
>
> It would be nice if we can generalize this behavior. We could then switch
> arbitrary set of filters in runtime fairly easily. WDYT?
>
> Trustin
I think DIRMINA-201 solves the efficiency issue I was referring to. The
only thing that would have to change to make ProtocolCodecFilter suit my
needs is that SimpleProtocolDecoderOut shouldn't queue messages but
rather forward them to the nextFilter right away as they are written.
Then my IoHandler would be able to instruct my Decoder instance to
change its state and decode differently depending on what the previous
message was.
I think it would be a bit more involved to also allow switching Decoder
instances completely between messages. But nothing is impossible... :)
/Niklas
Re: Switching codecs in runtime (Was: Removing synchronization on
a ProtocolEncoder and a ProtocolDecoder)
Posted by Michael Bauroth <mi...@falcom.de>.
Hi,
I would be interested in that piece of source. Could you post it as a
sample here?
Regards
Michael
Trustin Lee wrote:
> On 3/27/06, Niklas Therning <ni...@trillian.se> wrote:
>
>>In protocols like SMTP when there are simple line-based commands
>>intermixed with raw data (mail data) there are also great opportunities
>>for optimization if you write your own codec filter. I've implemented my
>>own DecoderFilter which can operate in "data mode". When not in data
>>mode the filter will act more or less like an ordinary ProtocolDecoder,
>>copying the received buffer to an autoexpanding buffer, decode SMTP
>>commands and pass them on to the next filter. When in data mode however
>>the filter will simply forward the buffers as they are received without
>>any copying (in most cases).
>>
>>I guess this could be achieved with the MINA codec package but not
>>without some tweaking and not as efficiently. Please let me know if I'm
>>wrong.
>>
>>I wouldn't mind adding this filter to MINA if anyone is interested.
>
>
>
> It would be nice if we can generalize this behavior. We could then switch
> arbitrary set of filters in runtime fairly easily. WDYT?
>
> Trustin
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
> --
> PGP key fingerprints:
> * E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
> * B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
>