You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Johannes Zillmann <JZ...@yahoo.de> on 2006/01/06 21:02:27 UTC
direct access to sockets
Dear mina team,
currently i'm participate in 2 projects in which network communication
plays a big role.
The profile of the projects looks similar:
- there is a lot of coordination communication to do, whereat mina is great
- but a lot of large data blocks(simply call it files..) has to be moved
as well, whereat mina naturally can't beat fe a simple bio socket
implementation in performance
After testing a lot and thinking some time about it, i came to the point
that the simplest way to gain the efforts of using
mina but not loosing any piece of performance is to have the possibilty
to get direct access to a sessions socket (no matter if bio or nio).
This would avoid the overhead of chunking, filtering,... data which is
not intended to do so.
And it would allow things like
FileChannel fc;
SocketChannel sc;
fc.transferFrom(sc,0,23);
Of cause it may break the api ?
What do you think about ?
best regards
Johannes
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
Re: direct access to sockets
Posted by Johannes Zillmann <JZ...@yahoo.de>.
>
>
> Sounds very reasonable. Fortunately, I also need this feature for my
> new project for the company. I believe it will be included in the
> future version of MINA somehow.
>
Wonderfull! :)
Johannes
> HTH,
> Trustin
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
> PGP Key ID: 0x854B996C
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
Re: direct access to sockets
Posted by Trustin Lee <tr...@gmail.com>.
2006/1/24, Johannes Zillmann <JZ...@yahoo.de>:
>
> Trustin Lee schrieb:
>
> >
> > Yes, we can do this using traffic control, but it breaks the
> > abstraction layer seriously. Does it worth? It would be great if you
> > give me good reasons.
>
> Well let mey i try it...
> First of all my requirements on a communication framework looks like :
> "transport" of (1) buissness logic & (2) large amount of data.
> For (1) mina looks quite good! nothing to add...
> (2) i achieved with some additional support classes also in a good
> manner, but compared with a bio socket transfer the performance is worser.
> Transfering a 80Mb file with mina is 1-2 seconds slower as with a
> simplistic bio implementation. (at least i achieved this results)
> Not remarkable i think, cause for copying a file. streaming is perfect,
> you just have nothing to add.
>
> I've doing some file copying test on my local machine without sockets
> and i was never able to achieve better performance with nio than with
> bio except with the transferTo() method.
> Searching the internet i came to the point i 've doing nothing wrong but
> i depends on the operating system too.
>
> Back to main point, i suspect loosing just one second on 80Mb in a
> scenario where giga bytes have to moved daily could be too much .
> It could put me in the situation where i cannot use mina but build a bio
> communication framework.
> But i think if i would do the job well, i end up in building a to mina
> similar framework for my buisness logic but with better support for
> streaming.
> Would that make sense ?
> I don't like the idea ...
>
> Well let me put it all together...
> If mina would support direct access to the sockets, i can be sure that i
> can achieve the best performance i ever could achieve!
> More then that, when you using mina you can take profit of mina, but
> would not loosing any possibilty of pure sockets,....
> Regardless if i use mina-nio or mina-bio.
> So i think the reasons not to use mina would then be frightening small !
>
> Sounds that reasonable ?
> I think there are a lot of applictaions out there that have to move
> large data masses, and i think especially them would profit the most!
Sounds very reasonable. Fortunately, I also need this feature for my new
project for the company. I believe it will be included in the future
version of MINA somehow.
HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
PGP Key ID: 0x854B996C
Re: direct access to sockets
Posted by Johannes Zillmann <JZ...@yahoo.de>.
Trustin Lee schrieb:
>
> Yes, we can do this using traffic control, but it breaks the
> abstraction layer seriously. Does it worth? It would be great if you
> give me good reasons.
Well let mey i try it...
First of all my requirements on a communication framework looks like :
"transport" of (1) buissness logic & (2) large amount of data.
For (1) mina looks quite good! nothing to add...
(2) i achieved with some additional support classes also in a good
manner, but compared with a bio socket transfer the performance is worser.
Transfering a 80Mb file with mina is 1-2 seconds slower as with a
simplistic bio implementation. (at least i achieved this results)
Not remarkable i think, cause for copying a file. streaming is perfect,
you just have nothing to add.
I've doing some file copying test on my local machine without sockets
and i was never able to achieve better performance with nio than with
bio except with the transferTo() method.
Searching the internet i came to the point i 've doing nothing wrong but
i depends on the operating system too.
Back to main point, i suspect loosing just one second on 80Mb in a
scenario where giga bytes have to moved daily could be too much .
It could put me in the situation where i cannot use mina but build a bio
communication framework.
But i think if i would do the job well, i end up in building a to mina
similar framework for my buisness logic but with better support for
streaming.
Would that make sense ?
I don't like the idea ...
Well let me put it all together...
If mina would support direct access to the sockets, i can be sure that i
can achieve the best performance i ever could achieve!
More then that, when you using mina you can take profit of mina, but
would not loosing any possibilty of pure sockets,....
Regardless if i use mina-nio or mina-bio.
So i think the reasons not to use mina would then be frightening small !
Sounds that reasonable ?
I think there are a lot of applictaions out there that have to move
large data masses, and i think especially them would profit the most!
wdyt?
Sorry, my english is not so good, especially today ;)
Johannes
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
Re: direct access to sockets
Posted by Trustin Lee <tr...@gmail.com>.
2006/1/13, Johannes Zillmann <JZ...@yahoo.de>:
>
> I'don't setp deep into nio or mina core yet,
> so is that possible ...
>
> If you haven a connected session and decide now to get directly acces to
> socketChannel
> -> tell the SocketIoProcessor that it should not perform any read/writes
> on the session socket channel (mayby just with setting a sessions
> attribute)
> -> retrieve the socketCjannle from session
> -> work with it
> -> maybe integrate it back to SocketIoProcessor io-work
>
> ?
Yes, we can do this using traffic control, but it breaks the abstraction
layer seriously. Does it worth? It would be great if you give me good
reasons.
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
PGP Key ID: 0x854B996C
Re: direct access to sockets
Posted by Johannes Zillmann <JZ...@yahoo.de>.
Trustin Lee schrieb:
> 2006/1/13, Johannes Zillmann <JZ8_blue@yahoo.de
> <ma...@yahoo.de>>:
>
> > For now there's no way to achieve the same behavior with
> this. But I
> > guess we can implement this by providing a special write request
> object:
> >
> > IoSession session = ...;
> > session.write( new TransferRequest( fc, 0, 23 ) );
> >
> > WDYT?
>
> Well i think there are 2 sides. The sending and the receiving... ;)
> For the sending part the above approach looks very nice !
> For the receiving part could you think of somthing like :
> socketSession.decoupleSocketChannel() -> SocketChannel
> ??
>
>
> Now I got it. You want to receive data from the channel directly.
> This is not possible because it will cause serious concurrency issue
> with SocketIoProcessor. All I/O has to happen in the worker thread
> that SocketIoProcessor manager. It you want to use blocking I/O for
> this, then you'll have to use StreamIoHandler for now. Please let me
> know if you have any good idea to resolve this issue.
I'don't setp deep into nio or mina core yet,
so is that possible ...
If you haven a connected session and decide now to get directly acces to
socketChannel
-> tell the SocketIoProcessor that it should not perform any read/writes
on the session socket channel (mayby just with setting a sessions attribute)
-> retrieve the socketCjannle from session
-> work with it
-> maybe integrate it back to SocketIoProcessor io-work
?
Johannes
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
Re: direct access to sockets
Posted by Trustin Lee <tr...@gmail.com>.
2006/1/13, Johannes Zillmann <JZ...@yahoo.de>:
>
> > For now there's no way to achieve the same behavior with this. But I
> > guess we can implement this by providing a special write request object:
> >
> > IoSession session = ...;
> > session.write( new TransferRequest( fc, 0, 23 ) );
> >
> > WDYT?
>
> Well i think there are 2 sides. The sending and the receiving... ;)
> For the sending part the above approach looks very nice !
> For the receiving part could you think of somthing like :
> socketSession.decoupleSocketChannel() -> SocketChannel
> ??
Now I got it. You want to receive data from the channel directly. This is
not possible because it will cause serious concurrency issue with
SocketIoProcessor. All I/O has to happen in the worker thread that
SocketIoProcessor manager. It you want to use blocking I/O for this, then
you'll have to use StreamIoHandler for now. Please let me know if you have
any good idea to resolve this issue.
Thanks,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
PGP Key ID: 0x854B996C
Re: direct access to sockets
Posted by Johannes Zillmann <JZ...@yahoo.de>.
Hello Trustin,
Trustin Lee schrieb:
> Hi Johannes,
>
> 2006/1/7, Johannes Zillmann <JZ8_blue@yahoo.de
> <ma...@yahoo.de>>:
>
> Dear mina team,
>
> currently i'm participate in 2 projects in which network communication
> plays a big role.
> The profile of the projects looks similar:
> - there is a lot of coordination communication to do, whereat mina
> is great
> - but a lot of large data blocks(simply call it files..) has to be
> moved
> as well, whereat mina naturally can't beat fe a simple bio socket
> implementation in performance
>
> After testing a lot and thinking some time about it, i came to the
> point
> that the simplest way to gain the efforts of using
> mina but not loosing any piece of performance is to have the
> possibilty
> to get direct access to a sessions socket (no matter if bio or nio).
>
> This would avoid the overhead of chunking, filtering,... data
> which is
> not intended to do so.
> And it would allow things like
> FileChannel fc;
> SocketChannel sc;
> fc.transferFrom(sc,0,23);
>
>
> For now there's no way to achieve the same behavior with this. But I
> guess we can implement this by providing a special write request object:
>
> IoSession session = ...;
> session.write( new TransferRequest( fc, 0, 23 ) );
>
> WDYT?
Well i think there are 2 sides. The sending and the receiving... ;)
For the sending part the above approach looks very nice !
For the receiving part could you think of somthing like :
socketSession.decoupleSocketChannel() -> SocketChannel
??
Johannes
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
Re: direct access to sockets
Posted by Trustin Lee <tr...@gmail.com>.
Hi Johannes,
2006/1/7, Johannes Zillmann <JZ...@yahoo.de>:
>
> Dear mina team,
>
> currently i'm participate in 2 projects in which network communication
> plays a big role.
> The profile of the projects looks similar:
> - there is a lot of coordination communication to do, whereat mina is
> great
> - but a lot of large data blocks(simply call it files..) has to be moved
> as well, whereat mina naturally can't beat fe a simple bio socket
> implementation in performance
>
> After testing a lot and thinking some time about it, i came to the point
> that the simplest way to gain the efforts of using
> mina but not loosing any piece of performance is to have the possibilty
> to get direct access to a sessions socket (no matter if bio or nio).
>
> This would avoid the overhead of chunking, filtering,... data which is
> not intended to do so.
> And it would allow things like
> FileChannel fc;
> SocketChannel sc;
> fc.transferFrom(sc,0,23);
For now there's no way to achieve the same behavior with this. But I guess
we can implement this by providing a special write request object:
IoSession session = ...;
session.write( new TransferRequest( fc, 0, 23 ) );
WDYT?
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
PGP Key ID: 0x854B996C
[MINA] Re: direct access to sockets
Posted by Johannes Zillmann <JZ...@yahoo.de>.
I'm soryy,
i've forgott the mina prefix...
Johannes Zillmann schrieb:
> Dear mina team,
>
> currently i'm participate in 2 projects in which network communication
> plays a big role.
> The profile of the projects looks similar:
> - there is a lot of coordination communication to do, whereat mina is
> great
> - but a lot of large data blocks(simply call it files..) has to be
> moved as well, whereat mina naturally can't beat fe a simple bio
> socket implementation in performance
>
> After testing a lot and thinking some time about it, i came to the
> point that the simplest way to gain the efforts of using
> mina but not loosing any piece of performance is to have the
> possibilty to get direct access to a sessions socket (no matter if bio
> or nio).
>
> This would avoid the overhead of chunking, filtering,... data which is
> not intended to do so.
> And it would allow things like
> FileChannel fc;
> SocketChannel sc;
> fc.transferFrom(sc,0,23);
>
> Of cause it may break the api ?
>
> What do you think about ?
>
> best regards
> Johannes
>
>
>
>
>
___________________________________________________________
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de