You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Alessandro Torrisi <al...@eurone.it> on 2005/12/27 10:07:25 UTC
[mina] Thread Pools, Concurrent connections and Performance
Hi ! I'm developing a free and opensource Direct Connect software (P2P
server). Now that protocol implementation is quite complete I'm testing
with lot of connections.
When connections are made in a concurrent way (50-100 a time), the
server seems to be blocked...
Can I do something to improve performance, adjusting some parameter or
applying some programming pattern directly on Mina ? I've listened about
Thread Pools on SocketAcceptor and IoThreadPool but I didn't find any
tutorial or documentation, is it possible to directly configure these ones ?
A C++ hubsoft (for example verlihub) can have over 10.000 concurrent
connections on a 100 Mbit line, on my personal line (10 Mbit) it can
have 2000-2500 users. I tested it with 600-700 connections, the software
hung up.
It could be that my software is not well optimized, but even if I
doubled the used bandwidth I can't see 1000 users !
I listened about Mina scalability and pointed on it...please help me !
What are the base step to introduce a new more performant architecture
without rewriting the code from scratch or upset the code ?
Best regards (and excuse me for my english),
Alessandro
Re: direct access to sockets
Posted by Johannes Zillmann <JZ...@yahoo.de>.
>
>
> Sounds very reasonable. Fortunately, I also need this feature for my
> new project for the company. I believe it will be included in the
> future version of MINA somehow.
>
Wonderfull! :)
Johannes
> HTH,
> Trustin
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
> PGP Key ID: 0x854B996C
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
Re: direct access to sockets
Posted by Trustin Lee <tr...@gmail.com>.
2006/1/24, Johannes Zillmann <JZ...@yahoo.de>:
>
> Trustin Lee schrieb:
>
> >
> > Yes, we can do this using traffic control, but it breaks the
> > abstraction layer seriously. Does it worth? It would be great if you
> > give me good reasons.
>
> Well let mey i try it...
> First of all my requirements on a communication framework looks like :
> "transport" of (1) buissness logic & (2) large amount of data.
> For (1) mina looks quite good! nothing to add...
> (2) i achieved with some additional support classes also in a good
> manner, but compared with a bio socket transfer the performance is worser.
> Transfering a 80Mb file with mina is 1-2 seconds slower as with a
> simplistic bio implementation. (at least i achieved this results)
> Not remarkable i think, cause for copying a file. streaming is perfect,
> you just have nothing to add.
>
> I've doing some file copying test on my local machine without sockets
> and i was never able to achieve better performance with nio than with
> bio except with the transferTo() method.
> Searching the internet i came to the point i 've doing nothing wrong but
> i depends on the operating system too.
>
> Back to main point, i suspect loosing just one second on 80Mb in a
> scenario where giga bytes have to moved daily could be too much .
> It could put me in the situation where i cannot use mina but build a bio
> communication framework.
> But i think if i would do the job well, i end up in building a to mina
> similar framework for my buisness logic but with better support for
> streaming.
> Would that make sense ?
> I don't like the idea ...
>
> Well let me put it all together...
> If mina would support direct access to the sockets, i can be sure that i
> can achieve the best performance i ever could achieve!
> More then that, when you using mina you can take profit of mina, but
> would not loosing any possibilty of pure sockets,....
> Regardless if i use mina-nio or mina-bio.
> So i think the reasons not to use mina would then be frightening small !
>
> Sounds that reasonable ?
> I think there are a lot of applictaions out there that have to move
> large data masses, and i think especially them would profit the most!
Sounds very reasonable. Fortunately, I also need this feature for my new
project for the company. I believe it will be included in the future
version of MINA somehow.
HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
PGP Key ID: 0x854B996C
Re: direct access to sockets
Posted by Johannes Zillmann <JZ...@yahoo.de>.
Trustin Lee schrieb:
>
> Yes, we can do this using traffic control, but it breaks the
> abstraction layer seriously. Does it worth? It would be great if you
> give me good reasons.
Well let mey i try it...
First of all my requirements on a communication framework looks like :
"transport" of (1) buissness logic & (2) large amount of data.
For (1) mina looks quite good! nothing to add...
(2) i achieved with some additional support classes also in a good
manner, but compared with a bio socket transfer the performance is worser.
Transfering a 80Mb file with mina is 1-2 seconds slower as with a
simplistic bio implementation. (at least i achieved this results)
Not remarkable i think, cause for copying a file. streaming is perfect,
you just have nothing to add.
I've doing some file copying test on my local machine without sockets
and i was never able to achieve better performance with nio than with
bio except with the transferTo() method.
Searching the internet i came to the point i 've doing nothing wrong but
i depends on the operating system too.
Back to main point, i suspect loosing just one second on 80Mb in a
scenario where giga bytes have to moved daily could be too much .
It could put me in the situation where i cannot use mina but build a bio
communication framework.
But i think if i would do the job well, i end up in building a to mina
similar framework for my buisness logic but with better support for
streaming.
Would that make sense ?
I don't like the idea ...
Well let me put it all together...
If mina would support direct access to the sockets, i can be sure that i
can achieve the best performance i ever could achieve!
More then that, when you using mina you can take profit of mina, but
would not loosing any possibilty of pure sockets,....
Regardless if i use mina-nio or mina-bio.
So i think the reasons not to use mina would then be frightening small !
Sounds that reasonable ?
I think there are a lot of applictaions out there that have to move
large data masses, and i think especially them would profit the most!
wdyt?
Sorry, my english is not so good, especially today ;)
Johannes
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
Re: direct access to sockets
Posted by Trustin Lee <tr...@gmail.com>.
2006/1/13, Johannes Zillmann <JZ...@yahoo.de>:
>
> I'don't setp deep into nio or mina core yet,
> so is that possible ...
>
> If you haven a connected session and decide now to get directly acces to
> socketChannel
> -> tell the SocketIoProcessor that it should not perform any read/writes
> on the session socket channel (mayby just with setting a sessions
> attribute)
> -> retrieve the socketCjannle from session
> -> work with it
> -> maybe integrate it back to SocketIoProcessor io-work
>
> ?
Yes, we can do this using traffic control, but it breaks the abstraction
layer seriously. Does it worth? It would be great if you give me good
reasons.
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
PGP Key ID: 0x854B996C
Re: direct access to sockets
Posted by Johannes Zillmann <JZ...@yahoo.de>.
Trustin Lee schrieb:
> 2006/1/13, Johannes Zillmann <JZ8_blue@yahoo.de
> <ma...@yahoo.de>>:
>
> > For now there's no way to achieve the same behavior with
> this. But I
> > guess we can implement this by providing a special write request
> object:
> >
> > IoSession session = ...;
> > session.write( new TransferRequest( fc, 0, 23 ) );
> >
> > WDYT?
>
> Well i think there are 2 sides. The sending and the receiving... ;)
> For the sending part the above approach looks very nice !
> For the receiving part could you think of somthing like :
> socketSession.decoupleSocketChannel() -> SocketChannel
> ??
>
>
> Now I got it. You want to receive data from the channel directly.
> This is not possible because it will cause serious concurrency issue
> with SocketIoProcessor. All I/O has to happen in the worker thread
> that SocketIoProcessor manager. It you want to use blocking I/O for
> this, then you'll have to use StreamIoHandler for now. Please let me
> know if you have any good idea to resolve this issue.
I'don't setp deep into nio or mina core yet,
so is that possible ...
If you haven a connected session and decide now to get directly acces to
socketChannel
-> tell the SocketIoProcessor that it should not perform any read/writes
on the session socket channel (mayby just with setting a sessions attribute)
-> retrieve the socketCjannle from session
-> work with it
-> maybe integrate it back to SocketIoProcessor io-work
?
Johannes
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
Re: direct access to sockets
Posted by Trustin Lee <tr...@gmail.com>.
2006/1/13, Johannes Zillmann <JZ...@yahoo.de>:
>
> > For now there's no way to achieve the same behavior with this. But I
> > guess we can implement this by providing a special write request object:
> >
> > IoSession session = ...;
> > session.write( new TransferRequest( fc, 0, 23 ) );
> >
> > WDYT?
>
> Well i think there are 2 sides. The sending and the receiving... ;)
> For the sending part the above approach looks very nice !
> For the receiving part could you think of somthing like :
> socketSession.decoupleSocketChannel() -> SocketChannel
> ??
Now I got it. You want to receive data from the channel directly. This is
not possible because it will cause serious concurrency issue with
SocketIoProcessor. All I/O has to happen in the worker thread that
SocketIoProcessor manager. It you want to use blocking I/O for this, then
you'll have to use StreamIoHandler for now. Please let me know if you have
any good idea to resolve this issue.
Thanks,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
PGP Key ID: 0x854B996C
Re: direct access to sockets
Posted by Johannes Zillmann <JZ...@yahoo.de>.
Hello Trustin,
Trustin Lee schrieb:
> Hi Johannes,
>
> 2006/1/7, Johannes Zillmann <JZ8_blue@yahoo.de
> <ma...@yahoo.de>>:
>
> Dear mina team,
>
> currently i'm participate in 2 projects in which network communication
> plays a big role.
> The profile of the projects looks similar:
> - there is a lot of coordination communication to do, whereat mina
> is great
> - but a lot of large data blocks(simply call it files..) has to be
> moved
> as well, whereat mina naturally can't beat fe a simple bio socket
> implementation in performance
>
> After testing a lot and thinking some time about it, i came to the
> point
> that the simplest way to gain the efforts of using
> mina but not loosing any piece of performance is to have the
> possibilty
> to get direct access to a sessions socket (no matter if bio or nio).
>
> This would avoid the overhead of chunking, filtering,... data
> which is
> not intended to do so.
> And it would allow things like
> FileChannel fc;
> SocketChannel sc;
> fc.transferFrom(sc,0,23);
>
>
> For now there's no way to achieve the same behavior with this. But I
> guess we can implement this by providing a special write request object:
>
> IoSession session = ...;
> session.write( new TransferRequest( fc, 0, 23 ) );
>
> WDYT?
Well i think there are 2 sides. The sending and the receiving... ;)
For the sending part the above approach looks very nice !
For the receiving part could you think of somthing like :
socketSession.decoupleSocketChannel() -> SocketChannel
??
Johannes
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
Re: direct access to sockets
Posted by Trustin Lee <tr...@gmail.com>.
Hi Johannes,
2006/1/7, Johannes Zillmann <JZ...@yahoo.de>:
>
> Dear mina team,
>
> currently i'm participate in 2 projects in which network communication
> plays a big role.
> The profile of the projects looks similar:
> - there is a lot of coordination communication to do, whereat mina is
> great
> - but a lot of large data blocks(simply call it files..) has to be moved
> as well, whereat mina naturally can't beat fe a simple bio socket
> implementation in performance
>
> After testing a lot and thinking some time about it, i came to the point
> that the simplest way to gain the efforts of using
> mina but not loosing any piece of performance is to have the possibilty
> to get direct access to a sessions socket (no matter if bio or nio).
>
> This would avoid the overhead of chunking, filtering,... data which is
> not intended to do so.
> And it would allow things like
> FileChannel fc;
> SocketChannel sc;
> fc.transferFrom(sc,0,23);
For now there's no way to achieve the same behavior with this. But I guess
we can implement this by providing a special write request object:
IoSession session = ...;
session.write( new TransferRequest( fc, 0, 23 ) );
WDYT?
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
PGP Key ID: 0x854B996C
[MINA] Re: direct access to sockets
Posted by Johannes Zillmann <JZ...@yahoo.de>.
I'm soryy,
i've forgott the mina prefix...
Johannes Zillmann schrieb:
> Dear mina team,
>
> currently i'm participate in 2 projects in which network communication
> plays a big role.
> The profile of the projects looks similar:
> - there is a lot of coordination communication to do, whereat mina is
> great
> - but a lot of large data blocks(simply call it files..) has to be
> moved as well, whereat mina naturally can't beat fe a simple bio
> socket implementation in performance
>
> After testing a lot and thinking some time about it, i came to the
> point that the simplest way to gain the efforts of using
> mina but not loosing any piece of performance is to have the
> possibilty to get direct access to a sessions socket (no matter if bio
> or nio).
>
> This would avoid the overhead of chunking, filtering,... data which is
> not intended to do so.
> And it would allow things like
> FileChannel fc;
> SocketChannel sc;
> fc.transferFrom(sc,0,23);
>
> Of cause it may break the api ?
>
> What do you think about ?
>
> best regards
> Johannes
>
>
>
>
>
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
direct access to sockets
Posted by Johannes Zillmann <JZ...@yahoo.de>.
Dear mina team,
currently i'm participate in 2 projects in which network communication
plays a big role.
The profile of the projects looks similar:
- there is a lot of coordination communication to do, whereat mina is great
- but a lot of large data blocks(simply call it files..) has to be moved
as well, whereat mina naturally can't beat fe a simple bio socket
implementation in performance
After testing a lot and thinking some time about it, i came to the point
that the simplest way to gain the efforts of using
mina but not loosing any piece of performance is to have the possibilty
to get direct access to a sessions socket (no matter if bio or nio).
This would avoid the overhead of chunking, filtering,... data which is
not intended to do so.
And it would allow things like
FileChannel fc;
SocketChannel sc;
fc.transferFrom(sc,0,23);
Of cause it may break the api ?
What do you think about ?
best regards
Johannes
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
Re: [mina] Thread Pools, Concurrent connections and Performance
Posted by Alessandro Torrisi <al...@eurone.it>.
I'm really happy for your great availability...here it is the
opensource...that's what I like and probably love.
Yes it's hosted at sourceforge ! The project name is JxHub and here it
is the url: http://www.sourceforge.net/projects/jxhub
Most of the protocol is implemented, it seems to go well..but you know
the rest of the story...
Tell me if you want to access the cvs repository with other permissions.
Meanwhile I'm going to try the snippet for 0.9.x ... (I'm using Mina
Snapshot 0.9.1 I tryied this for the bind problem on Unix, I wrote also
for this one in the mailing list...I have to wait a minute or more for
port unbind ! But this is the minor problem for now...)
Best regards Niklas
Bye, Alex
Niklas Therning wrote:
> Alessandro Torrisi wrote:
>
>> IoThreadPoolFilter f = (IoThreadPoolFilter)
>> reg.getIoAcceptor(TransportType.SOCKET).getFilterChain().get("threadPool");
>>
>> f.setMaximumPoolSize(10);
>>
>> I discovered that this piece of code gives me
>> "NullPointerException", I have no FilterChain applied to the Acceptor
>> and in my Eclipse debugger I can see:
>
>
> From what I can tell from the code you have included below you are
> using MINA 0.9. As I explained in my previous mail the code above will
> only work in MINA 0.8 (in 0.9 you will get an NPE since f will be
> null). In MINA 0.9 you can try to to extend SimpleServiceRegistry
> instead as I described earlier:
>
> public class MyServiceRegistry extends SimpleServiceRegistry {
> public MyServiceRegistry() {
> super();
> threadPoolFilter.setMaximumPoolSize(10);
> }
> }
>
> Please note that the most important thing for scalability is that you
> never perform any blocking operations in your IoHandler
> implementation. Or at least make sure that they only block for a very
> short time. The code samples you included do not give enough
> information on whether you have any blocking operations.
>
>> ...
>>
>> I'm stopped ! I don't know where to look. I think there's too little
>> documentation or too few examples for Mina, I think more complex
>> samples are needed !
>> Regards, Alex
>
>
> Yes, you may be right about that. But your open source DC server will
> be an excellent example of a more complex MINA application. In the
> meantime the MINA community is here to help you out.
>
> BTW, is your open source project hosted somewhere publicly like on
> sourceforge? If it is I could have a look if you want to.
>
> /Niklas
>
>>
>> Niklas Therning wrote:
>>
>>> No, the number of threads in your thread pool does not limit the
>>> number of concurrent clients. That's actually one of the big
>>> benefits of using MINA. Since everything is event based you can use
>>> a small number of threads to handle a larger number of connections.
>>>
>>> Please, try to set the max pool size to a small number (5-10 maybe)
>>> and see if this has any impact on the performance.
>>>
>>> /Niklas
>>>
>>> Alessandro Torrisi wrote:
>>>
>>>> But in this way I will limit users max number in the hub server I
>>>> think...
>>>> Is there any way to have a significant increase of performance ?
>>>>
>>>> Regards,
>>>> Alex.
>>>>
>>>>
>>>> Niklas Therning wrote:
>>>>
>>>>> Alessandro Torrisi wrote:
>>>>>
>>>>>> Hi ! I'm developing a free and opensource Direct Connect software
>>>>>> (P2P server). Now that protocol implementation is quite complete
>>>>>> I'm testing with lot of connections.
>>>>>> When connections are made in a concurrent way (50-100 a time),
>>>>>> the server seems to be blocked...
>>>>>> Can I do something to improve performance, adjusting some
>>>>>> parameter or applying some programming pattern directly on Mina ?
>>>>>> I've listened about Thread Pools on SocketAcceptor and
>>>>>> IoThreadPool but I didn't find any tutorial or documentation, is
>>>>>> it possible to directly configure these ones ?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Yes. By default the maximum thread pool size equals
>>>>> Integer.MAX_VALUE. It can be changed but this has changed bewteen
>>>>> MINA 0.8 and 0.9.
>>>>>
>>>>> In 0.8, when using SimpleServiceRegistry, you can configure the
>>>>> maximum pool size like this:
>>>>>
>>>>> IoThreadPoolFilter f = (IoThreadPoolFilter)
>>>>> reg.getIoAcceptor(TransportType.SOCKET).getFilterChain().get("threadPool");
>>>>>
>>>>> f.setMaximumPoolSize(10);
>>>>>
>>>>> In 0.9 its not that easy since the ThreadPoolFilter used by
>>>>> SimpleServiceRegistry isn't accessible until a session has been
>>>>> created. You could try to extend SimpleServiceRegistry and
>>>>> configure the protected threadPoolFilter yourself:
>>>>>
>>>>> public class MyServiceRegistry extends SimpleServiceRegistry {
>>>>> public MyServiceRegistry() {
>>>>> super();
>>>>> threadPoolFilter.setMaximumPoolSize(10);
>>>>> }
>>>>> }
>>>>>
>>>>> And then instead of using SimpleServiceRegistry you use
>>>>> MyServiceRegistry.
>>>>>
>>>>> Both of these approaches will use a thread pool of at most 10
>>>>> threads.
>>>>>
>>>>> HTH
>>>>> /Niklas
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>
Re: [mina] Thread Pools, Concurrent connections and Performance
Posted by Niklas Therning <ni...@trillian.se>.
Alessandro Torrisi wrote:
> IoThreadPoolFilter f = (IoThreadPoolFilter)
> reg.getIoAcceptor(TransportType.SOCKET).getFilterChain().get("threadPool");
> f.setMaximumPoolSize(10);
>
> I discovered that this piece of code gives me "NullPointerException", I
> have no FilterChain applied to the Acceptor and in my Eclipse debugger I
> can see:
From what I can tell from the code you have included below you are
using MINA 0.9. As I explained in my previous mail the code above will
only work in MINA 0.8 (in 0.9 you will get an NPE since f will be null).
In MINA 0.9 you can try to to extend SimpleServiceRegistry instead as I
described earlier:
public class MyServiceRegistry extends SimpleServiceRegistry {
public MyServiceRegistry() {
super();
threadPoolFilter.setMaximumPoolSize(10);
}
}
Please note that the most important thing for scalability is that you
never perform any blocking operations in your IoHandler implementation.
Or at least make sure that they only block for a very short time. The
code samples you included do not give enough information on whether you
have any blocking operations.
> ...
>
> I'm stopped ! I don't know where to look. I think there's too little
> documentation or too few examples for Mina, I think more complex samples
> are needed !
> Regards, Alex
Yes, you may be right about that. But your open source DC server will be
an excellent example of a more complex MINA application. In the meantime
the MINA community is here to help you out.
BTW, is your open source project hosted somewhere publicly like on
sourceforge? If it is I could have a look if you want to.
/Niklas
>
> Niklas Therning wrote:
>
>> No, the number of threads in your thread pool does not limit the
>> number of concurrent clients. That's actually one of the big benefits
>> of using MINA. Since everything is event based you can use a small
>> number of threads to handle a larger number of connections.
>>
>> Please, try to set the max pool size to a small number (5-10 maybe)
>> and see if this has any impact on the performance.
>>
>> /Niklas
>>
>> Alessandro Torrisi wrote:
>>
>>> But in this way I will limit users max number in the hub server I
>>> think...
>>> Is there any way to have a significant increase of performance ?
>>>
>>> Regards,
>>> Alex.
>>>
>>>
>>> Niklas Therning wrote:
>>>
>>>> Alessandro Torrisi wrote:
>>>>
>>>>> Hi ! I'm developing a free and opensource Direct Connect software
>>>>> (P2P server). Now that protocol implementation is quite complete
>>>>> I'm testing with lot of connections.
>>>>> When connections are made in a concurrent way (50-100 a time), the
>>>>> server seems to be blocked...
>>>>> Can I do something to improve performance, adjusting some parameter
>>>>> or applying some programming pattern directly on Mina ? I've
>>>>> listened about Thread Pools on SocketAcceptor and IoThreadPool but
>>>>> I didn't find any tutorial or documentation, is it possible to
>>>>> directly configure these ones ?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Yes. By default the maximum thread pool size equals
>>>> Integer.MAX_VALUE. It can be changed but this has changed bewteen
>>>> MINA 0.8 and 0.9.
>>>>
>>>> In 0.8, when using SimpleServiceRegistry, you can configure the
>>>> maximum pool size like this:
>>>>
>>>> IoThreadPoolFilter f = (IoThreadPoolFilter)
>>>> reg.getIoAcceptor(TransportType.SOCKET).getFilterChain().get("threadPool");
>>>>
>>>> f.setMaximumPoolSize(10);
>>>>
>>>> In 0.9 its not that easy since the ThreadPoolFilter used by
>>>> SimpleServiceRegistry isn't accessible until a session has been
>>>> created. You could try to extend SimpleServiceRegistry and configure
>>>> the protected threadPoolFilter yourself:
>>>>
>>>> public class MyServiceRegistry extends SimpleServiceRegistry {
>>>> public MyServiceRegistry() {
>>>> super();
>>>> threadPoolFilter.setMaximumPoolSize(10);
>>>> }
>>>> }
>>>>
>>>> And then instead of using SimpleServiceRegistry you use
>>>> MyServiceRegistry.
>>>>
>>>> Both of these approaches will use a thread pool of at most 10 threads.
>>>>
>>>> HTH
>>>> /Niklas
>>>
>>>
>>>
>>>
>>
Re: [mina] Thread Pools, Concurrent connections and Performance
Posted by Alessandro Torrisi <al...@eurone.it>.
IoThreadPoolFilter f = (IoThreadPoolFilter)
reg.getIoAcceptor(TransportType.SOCKET).getFilterChain().get("threadPool");
f.setMaximumPoolSize(10);
I discovered that this piece of code gives me "NullPointerException", I
have no FilterChain applied to the Acceptor and in my Eclipse debugger I
can see:
"registry.getAcceptor(TransportType.SOCKET).getFilterChain()" is null
probably because I didn't create any filter...is it right ? And if not
is there any tutorial which explain the use of the Filters or Codecs (I
don't use this, I simply created a class which extends IoHandlerAdapter
and from there I have overriden the methods I wanted to use like
sessionOpened(...) sessionClosed(...) onMessageReceived(...) and from
here I started to implement my high level protocol (Direct Connect)
Here it is a little snippet which can make you understand my probably
wrong impact on the software:
@Override
public void messageReceived(IoSession session, Object obj) throws
Exception {
// I get the message
String msg = null;
try {
// Object is a ByteBuffer
ByteBuffer buf = (ByteBuffer) obj;
// I get the string from the ByteBuffer decoding with
Windows-1252 charset decoder
msg = buf.getString(decoderCp1252);
log.debug("Received message was " + msg);
} catch (Exception _e) {
// msg is null
_e.printStackTrace();
}
// load plugins and execute
boolean process = true;
for (JxHubAbstractPlugin plugin : pluginManager.getPlugins()) {
boolean tmp = plugin.onMessageReceived(session, msg);
if (!tmp)
process = false;
}
if (process)
// And finally here it is where I call the method which will
dispatch my command
// pipeline is a JxHubPipeline a object I have created and that I
report after this snippet
pipeline.dispatchAction(session, msg);
}
// Here it is the JxHubPipeline structure
JxHubPipeline(JxHubDbPoolFactory, ConcurrentHashMap<String, IoSession>,
JxHubConfiguration, JxHubPluginManager)
// it will understand what kind of message it is and it will forward to
the right method
dispatchAction(IoSession, String)
processUserIp(IoSession, String)
processOpForceMove(IoSession, String)
processGetInfo(IoSession, String)
processConnectToMe(IoSession, String)
processMyInfo(IoSession, String)
processGetNickList(IoSession, String)
processSupports(IoSession, String)
processKey(IoSession, String)
processChat(IoSession, String)
processValidateNick(IoSession, String)
processOperatorCommand(IoSession, String)
processMyPass(IoSession, String)
processRevConnectToMe(IoSession, String)
processTo(IoSession, String)
processSearch(IoSession, String)
processSr(IoSession, String)
I'm stopped ! I don't know where to look. I think there's too little
documentation or too few examples for Mina, I think more complex samples
are needed !
Regards, Alex
Niklas Therning wrote:
> No, the number of threads in your thread pool does not limit the
> number of concurrent clients. That's actually one of the big benefits
> of using MINA. Since everything is event based you can use a small
> number of threads to handle a larger number of connections.
>
> Please, try to set the max pool size to a small number (5-10 maybe)
> and see if this has any impact on the performance.
>
> /Niklas
>
> Alessandro Torrisi wrote:
>
>> But in this way I will limit users max number in the hub server I
>> think...
>> Is there any way to have a significant increase of performance ?
>>
>> Regards,
>> Alex.
>>
>>
>> Niklas Therning wrote:
>>
>>> Alessandro Torrisi wrote:
>>>
>>>> Hi ! I'm developing a free and opensource Direct Connect software
>>>> (P2P server). Now that protocol implementation is quite complete
>>>> I'm testing with lot of connections.
>>>> When connections are made in a concurrent way (50-100 a time), the
>>>> server seems to be blocked...
>>>> Can I do something to improve performance, adjusting some parameter
>>>> or applying some programming pattern directly on Mina ? I've
>>>> listened about Thread Pools on SocketAcceptor and IoThreadPool but
>>>> I didn't find any tutorial or documentation, is it possible to
>>>> directly configure these ones ?
>>>
>>>
>>>
>>>
>>> Yes. By default the maximum thread pool size equals
>>> Integer.MAX_VALUE. It can be changed but this has changed bewteen
>>> MINA 0.8 and 0.9.
>>>
>>> In 0.8, when using SimpleServiceRegistry, you can configure the
>>> maximum pool size like this:
>>>
>>> IoThreadPoolFilter f = (IoThreadPoolFilter)
>>> reg.getIoAcceptor(TransportType.SOCKET).getFilterChain().get("threadPool");
>>>
>>> f.setMaximumPoolSize(10);
>>>
>>> In 0.9 its not that easy since the ThreadPoolFilter used by
>>> SimpleServiceRegistry isn't accessible until a session has been
>>> created. You could try to extend SimpleServiceRegistry and configure
>>> the protected threadPoolFilter yourself:
>>>
>>> public class MyServiceRegistry extends SimpleServiceRegistry {
>>> public MyServiceRegistry() {
>>> super();
>>> threadPoolFilter.setMaximumPoolSize(10);
>>> }
>>> }
>>>
>>> And then instead of using SimpleServiceRegistry you use
>>> MyServiceRegistry.
>>>
>>> Both of these approaches will use a thread pool of at most 10 threads.
>>>
>>> HTH
>>> /Niklas
>>
>>
>>
>
Re: [mina] Thread Pools, Concurrent connections and Performance
Posted by Niklas Therning <ni...@trillian.se>.
No, the number of threads in your thread pool does not limit the number
of concurrent clients. That's actually one of the big benefits of using
MINA. Since everything is event based you can use a small number of
threads to handle a larger number of connections.
Please, try to set the max pool size to a small number (5-10 maybe) and
see if this has any impact on the performance.
/Niklas
Alessandro Torrisi wrote:
> But in this way I will limit users max number in the hub server I think...
> Is there any way to have a significant increase of performance ?
>
> Regards,
> Alex.
>
>
> Niklas Therning wrote:
>
>> Alessandro Torrisi wrote:
>>
>>> Hi ! I'm developing a free and opensource Direct Connect software
>>> (P2P server). Now that protocol implementation is quite complete I'm
>>> testing with lot of connections.
>>> When connections are made in a concurrent way (50-100 a time), the
>>> server seems to be blocked...
>>> Can I do something to improve performance, adjusting some parameter
>>> or applying some programming pattern directly on Mina ? I've listened
>>> about Thread Pools on SocketAcceptor and IoThreadPool but I didn't
>>> find any tutorial or documentation, is it possible to directly
>>> configure these ones ?
>>
>>
>>
>> Yes. By default the maximum thread pool size equals Integer.MAX_VALUE.
>> It can be changed but this has changed bewteen MINA 0.8 and 0.9.
>>
>> In 0.8, when using SimpleServiceRegistry, you can configure the
>> maximum pool size like this:
>>
>> IoThreadPoolFilter f = (IoThreadPoolFilter)
>> reg.getIoAcceptor(TransportType.SOCKET).getFilterChain().get("threadPool");
>>
>> f.setMaximumPoolSize(10);
>>
>> In 0.9 its not that easy since the ThreadPoolFilter used by
>> SimpleServiceRegistry isn't accessible until a session has been
>> created. You could try to extend SimpleServiceRegistry and configure
>> the protected threadPoolFilter yourself:
>>
>> public class MyServiceRegistry extends SimpleServiceRegistry {
>> public MyServiceRegistry() {
>> super();
>> threadPoolFilter.setMaximumPoolSize(10);
>> }
>> }
>>
>> And then instead of using SimpleServiceRegistry you use
>> MyServiceRegistry.
>>
>> Both of these approaches will use a thread pool of at most 10 threads.
>>
>> HTH
>> /Niklas
>
>
Re: [mina] Thread Pools, Concurrent connections and Performance
Posted by Alessandro Torrisi <al...@eurone.it>.
But in this way I will limit users max number in the hub server I think...
Is there any way to have a significant increase of performance ?
Regards,
Alex.
Niklas Therning wrote:
> Alessandro Torrisi wrote:
>
>> Hi ! I'm developing a free and opensource Direct Connect software
>> (P2P server). Now that protocol implementation is quite complete I'm
>> testing with lot of connections.
>> When connections are made in a concurrent way (50-100 a time), the
>> server seems to be blocked...
>> Can I do something to improve performance, adjusting some parameter
>> or applying some programming pattern directly on Mina ? I've listened
>> about Thread Pools on SocketAcceptor and IoThreadPool but I didn't
>> find any tutorial or documentation, is it possible to directly
>> configure these ones ?
>
>
> Yes. By default the maximum thread pool size equals Integer.MAX_VALUE.
> It can be changed but this has changed bewteen MINA 0.8 and 0.9.
>
> In 0.8, when using SimpleServiceRegistry, you can configure the
> maximum pool size like this:
>
> IoThreadPoolFilter f = (IoThreadPoolFilter)
> reg.getIoAcceptor(TransportType.SOCKET).getFilterChain().get("threadPool");
>
> f.setMaximumPoolSize(10);
>
> In 0.9 its not that easy since the ThreadPoolFilter used by
> SimpleServiceRegistry isn't accessible until a session has been
> created. You could try to extend SimpleServiceRegistry and configure
> the protected threadPoolFilter yourself:
>
> public class MyServiceRegistry extends SimpleServiceRegistry {
> public MyServiceRegistry() {
> super();
> threadPoolFilter.setMaximumPoolSize(10);
> }
> }
>
> And then instead of using SimpleServiceRegistry you use
> MyServiceRegistry.
>
> Both of these approaches will use a thread pool of at most 10 threads.
>
> HTH
> /Niklas
Re: [mina] Thread Pools, Concurrent connections and Performance
Posted by Niklas Therning <ni...@trillian.se>.
Alessandro Torrisi wrote:
> Hi ! I'm developing a free and opensource Direct Connect software (P2P
> server). Now that protocol implementation is quite complete I'm testing
> with lot of connections.
> When connections are made in a concurrent way (50-100 a time), the
> server seems to be blocked...
> Can I do something to improve performance, adjusting some parameter or
> applying some programming pattern directly on Mina ? I've listened about
> Thread Pools on SocketAcceptor and IoThreadPool but I didn't find any
> tutorial or documentation, is it possible to directly configure these
> ones ?
Yes. By default the maximum thread pool size equals Integer.MAX_VALUE.
It can be changed but this has changed bewteen MINA 0.8 and 0.9.
In 0.8, when using SimpleServiceRegistry, you can configure the maximum
pool size like this:
IoThreadPoolFilter f = (IoThreadPoolFilter)
reg.getIoAcceptor(TransportType.SOCKET).getFilterChain().get("threadPool");
f.setMaximumPoolSize(10);
In 0.9 its not that easy since the ThreadPoolFilter used by
SimpleServiceRegistry isn't accessible until a session has been created.
You could try to extend SimpleServiceRegistry and configure the
protected threadPoolFilter yourself:
public class MyServiceRegistry extends SimpleServiceRegistry {
public MyServiceRegistry() {
super();
threadPoolFilter.setMaximumPoolSize(10);
}
}
And then instead of using SimpleServiceRegistry you use MyServiceRegistry.
Both of these approaches will use a thread pool of at most 10 threads.
HTH
/Niklas