You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Mark <el...@gmail.com> on 2006/08/01 14:42:35 UTC
MINA FTP Server
I was looking into writing an FTP server using MINA. I found code in the
Apache incubator that I was going to use, but not sure if there are any
logistical issues that may come back to bite me if I use code in the
incubator. The FTP Server code seems to have good command handling and all,
so I think it makes sense to use something that has been tested.
What are peoples thoughts on this?
Thanks
Mark
Re: Logging
Posted by Trustin Lee <tr...@gmail.com>.
Hi Michael,
On 8/23/06, Michael Bauroth <mi...@falcom.de> wrote:
>
> Hi,
>
> Mina currently uses the SLF4J logging facade framework, right?
> Additionally I can choose, which underlying logging framework I want to
> use (log4j 1.3 in my case). Also right?
>
> How can I change now one of my logger levels at runtime to get temporary
> more or less detailed log messages? Slf4j doesn't support the setLevel()
> method in it's logger. It would be great, if somebody could me tell the
> right (Mina correct) way.
You could use SLF4J-log4j and do the exactly same thing as what you used to
do with Log4J because you're using Log4J actually.
HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: MINA blocks with unbindAll
Posted by fr...@free.fr.
Hi Trustin,
> Hi Fred,
>
> Finding deadlock or blocking problem is not really easy. Could you open a
> JIRA issue which attaches a full thread dump of the frozen server?
>
You're right about the difficulty. I just opened a JIRA issue.
For the thread dump, I use jdb since I am using Windows right
now (even if the program will later on runs on Unix) on my
laptop.
I hope this can help you.
>
> Thank you for using MINA, too! Any feedbacks are welcome!
> Trustin
I will surely come back with other questions or feedbacks
when I will start the qualification test...
Thanks a lot !
Frederic
Re: MINA blocks with unbindAll
Posted by Trustin Lee <tr...@gmail.com>.
Hi Fred,
Finding deadlock or blocking problem is not really easy. Could you open a
JIRA issue which attaches a full thread dump of the frozen server?
On 8/23/06, fred.bregier@free.fr <fr...@free.fr> wrote:
>
> Well, perhaps I didn't use the correct way to signal
> what I found as a bug or I didn't put the necessary
> information. So I redo my post and try to get to the
> essential. I am sure it is my fault since I surely
> badly explain my problem.
>
> I wanted to offer a function to shutdown gracefully the server,
> that is to say that the server doesn't accept new connection,
> but does let already connected client to finished correctly
> their session (which are multi messages related).
> In the beginning I used:
> - setDisconnectOnUnbind(false)
> - unbindAll()
> - wait in my business logic that all already connected users shutting down
> - setDisconnectOnUnbind(true)
> - unbindAll()
>
> I've got a lot of freeze (using 6 clients making a lot of messages)
> in either the first or the second unbindAll().
>
> So I simplify like this:
> - I am waiting in my business logic that all already connected users
> shutting
> down and does not accept new connection by hand (business logic).
> To make that, when I have the global static boolean shutdown as True,
> every new connexion are immediately closed by the server.
> Also, every pending connexion will finished their session, and when
> finished, the server also closed the connexion.
> The session used to send the shutdown order is closed immediately
> after receiving the order.
> - setDisconnectOnUnbind(true)
> - unbindAll()
>
> I still have easily a freeze in the server during the call to
> unbindAll().
>
> - If I have only 1 client, unbindAll is never blocking.
>
> - If I have 3 to 6 clients, making their stuff, they all disconnected
> correctly from the server (the server is closing the session itself).
> I saw that every clients are getting correctly the closed event.
> But the server often blocks during the call of unbindAll().
>
> I am no able at this time to see where is the problem in MINA.
> Everything seems ok for me from the point of vue of a programmer
> using MINA. But I surely make some mistakes since it seems
> that I am the only one to have this problem.
> Could it be related to the fact that this is the server that is
> forcing the close of the session ?
> The only thing I can tell now is that it is blocling during the
> call of unbindAll() in the server part.
>
> Hope this can help to see where the problem is.
>
> However, MINA is really a great job and I search about 3 month to
> find such a layer ! So thanks to all !
Thank you for using MINA, too! Any feedbacks are welcome!
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Logging
Posted by Michael Bauroth <mi...@falcom.de>.
Hi,
Mina currently uses the SLF4J logging facade framework, right?
Additionally I can choose, which underlying logging framework I want to
use (log4j 1.3 in my case). Also right?
How can I change now one of my logger levels at runtime to get temporary
more or less detailed log messages? Slf4j doesn't support the setLevel()
method in it's logger. It would be great, if somebody could me tell the
right (Mina correct) way.
Thanx in advance
Michael
Re: MINA blocks with unbindAll
Posted by fr...@free.fr.
Well, perhaps I didn't use the correct way to signal
what I found as a bug or I didn't put the necessary
information. So I redo my post and try to get to the
essential. I am sure it is my fault since I surely
badly explain my problem.
I wanted to offer a function to shutdown gracefully the server,
that is to say that the server doesn't accept new connection,
but does let already connected client to finished correctly
their session (which are multi messages related).
In the beginning I used:
- setDisconnectOnUnbind(false)
- unbindAll()
- wait in my business logic that all already connected users shutting down
- setDisconnectOnUnbind(true)
- unbindAll()
I've got a lot of freeze (using 6 clients making a lot of messages)
in either the first or the second unbindAll().
So I simplify like this:
- I am waiting in my business logic that all already connected users shutting
down and does not accept new connection by hand (business logic).
To make that, when I have the global static boolean shutdown as True,
every new connexion are immediately closed by the server.
Also, every pending connexion will finished their session, and when
finished, the server also closed the connexion.
The session used to send the shutdown order is closed immediately
after receiving the order.
- setDisconnectOnUnbind(true)
- unbindAll()
I still have easily a freeze in the server during the call to
unbindAll().
- If I have only 1 client, unbindAll is never blocking.
- If I have 3 to 6 clients, making their stuff, they all disconnected
correctly from the server (the server is closing the session itself).
I saw that every clients are getting correctly the closed event.
But the server often blocks during the call of unbindAll().
I am no able at this time to see where is the problem in MINA.
Everything seems ok for me from the point of vue of a programmer
using MINA. But I surely make some mistakes since it seems
that I am the only one to have this problem.
Could it be related to the fact that this is the server that is
forcing the close of the session ?
The only thing I can tell now is that it is blocling during the
call of unbindAll() in the server part.
Hope this can help to see where the problem is.
However, MINA is really a great job and I search about 3 month to
find such a layer ! So thanks to all !
Frederic
MINA blocks with unbindAll
Posted by fr...@free.fr.
Hi all !
I am really happy with MINA. It's a must have in our free devel!
I come back after a while getting more in development process
using MINA and I have some comments and one problem (see title).
Comments:
- I found that the memory issue I got before was a bit relative to the number
of new session created. The more I have new session, even if they closed
very quickly, the more the server and the client use memory.
It is not linear to the number of session opened during time but
the impact can be reproduced easily.
And it seems that when everything is finished (client waiting for
nothing in a big sleep and server waiting for new session), the
memory is not going really down.
For instance (under Windows testing) : All processes starting with 18Mb
- 8 sessions with each 400 messages in both ways => around 40 Mb
After a while (MINA Garbage collector) -> around 36Mb
- 3200 sessions (with each 1 message in both ways) => around 90 Mb
After a while (MINA GC) -> around 80 Mb
Everything is similar except the way session are done
(in first part, 1 session for many messages, in second, 1 session/message).
I don't knwo precisely why but it seems related to the high number
of sessions...
- In order to have a close.join() feature according to my desire,
I have implemented on top of MINA a "business" way:
- for my particular needs, I have some data to check if the session
is ready to be closed
- when I want to close a session, I first wait that all scheduledWriteRequest
is 0 then I close.
Now my problem I found in MINA unbindAll():
==========================================
I wanted to offer a function to shutdown gracefully the server,
that is to say that the server doesn't accept new connection,
but does let already connected client to finished correctly
their session (which are multi messages related).
In the beginning I used:
- setDisconnectOnUnbind(false)
- unbindAll()
- wait in my business logic that all already connected users shutting down
- setDisconnectOnUnbind(true)
- unbindAll()
I've got a lot of freeze (using 6 clients making a lot of messages)
in either the first or the second unbindAll().
So I simplify like this:
- wait in my business logic that all already connected users shutting down
and does not accept new connection by hand (business logic)
- setDisconnectOnUnbind(true)
- unbindAll()
And I still have easily a freeze in the server during the call to
unbindAll().
If I have only 1 client, unbindAll is not blocking (up to now).
If I have 3 to 6 clients, making their stuff, they all disconnected
correctly from the server (to make them disconnected, either
they receive a message that said "End of Session" so they closed,
or the server is closing the session itself.
But the server often blocks in the call of unbindAll().
Note that I use a different acceptor to get the server shutdown,
so the unbindAll is not related to the session used to get it down.
I am no able at this time to see where is the problem in MINA.
The only thing I can tell now is that it is blocling during the
call of unbindAll().
Hope this can help.
However, great job !
Frederic
Re: MINA FTP Server
Posted by Niklas Therning <ni...@trillian.se>.
To write back a binary file to the client use the StreamWriterFilter.
You must add it to the session's filter chain after your codec filter:
private static IoFilter CODEC_FILTER = new ProtocolCodecFilter(new
TextLineCodecFactory());
private static IoFilter STREAM_WRITE_FILTER = new StreamWriteFilter();
...
session.getFilterChain().addLast("codec", CODEC_FILTER);
session.getFilterChain().addLast("streamWriter", STREAM_WRITE_FILTER);
With the StreamWriteFilter in place you can write InputStreams directly
to the session. StreamWriteFilter will read the bytes from the stream
and pass those bytes along down the filter chain as MINA ByteBuffers. As
long as you write text or other messages which aren't InputStream
objects the StreamWriteFilter won't do anything. It's important that you
add StreamWriteFilter after your codec. ProtocolCodecFilter will not try
to encode anything written to the session which already is in byte form
(ByteBuffer).
Using the StreamWriteFilter like this all you have to do to write a file
to the client is:
session.write(new FileInputStream("c:\foo.txt"));
MINA will take care of the rest for you. If you need to do any
processing of the file data (like sending in it TEXT) mode you could
create a wrapper InputStream which wraps the FileInputStream and does
the processing in the read() method.
HTH
Niklas
Mark wrote:
> Great info. What I am looking for is a way to read in text (FTP
> Commands),
> and send text (responses) and binary (requested files). What I currently
> have is a protocol handler that extends IoHandlerAdapter. This class
> sets
> up a session filter using the following command:
>
> private static IoFilter CODEC_FILTER = new ProtocolCodecFilter(new
> TextLineCodecFactory());
> ...
> session.getFilterChain().addLast("codec", CODEC_FILTER);
>
> The problem is, how can I get my protocol handler to write a file back to
> the client? The only thing that I have found is to use a
> StreamIoHandler,
> but then I cannot figure out how to read the FTP commands.
>
> Thanks.
>
>
> On 8/20/06, peter royal <pr...@apache.org> wrote:
>>
>> On Aug 17, 2006, at 11:47 PM, Mark wrote:
>> > Basically, by the time the FTP Server is done, I will have to
>> > support text
>> > and binary over the same connection, and support sessions. Trying
>> > to figure
>> > out what is the best I/O class for this.
>> >
>> > ideas??
>>
>> Sessions are the MINA IoSession.
>>
>> For text and binary.. There's probably some sort of state-machine on
>> the server side? Both are just bytes to MINA, really.
>>
>> -pete
>>
>>
>> --
>> proyal@apache.org - http://fotap.org/~osi
>>
>>
>>
>>
>>
>>
>
--
Niklas Therning
www.spamdrain.net
Re: MINA FTP Server
Posted by Mark <el...@gmail.com>.
Great info. What I am looking for is a way to read in text (FTP Commands),
and send text (responses) and binary (requested files). What I currently
have is a protocol handler that extends IoHandlerAdapter. This class sets
up a session filter using the following command:
private static IoFilter CODEC_FILTER = new ProtocolCodecFilter(new
TextLineCodecFactory());
...
session.getFilterChain().addLast("codec", CODEC_FILTER);
The problem is, how can I get my protocol handler to write a file back to
the client? The only thing that I have found is to use a StreamIoHandler,
but then I cannot figure out how to read the FTP commands.
Thanks.
On 8/20/06, peter royal <pr...@apache.org> wrote:
>
> On Aug 17, 2006, at 11:47 PM, Mark wrote:
> > Basically, by the time the FTP Server is done, I will have to
> > support text
> > and binary over the same connection, and support sessions. Trying
> > to figure
> > out what is the best I/O class for this.
> >
> > ideas??
>
> Sessions are the MINA IoSession.
>
> For text and binary.. There's probably some sort of state-machine on
> the server side? Both are just bytes to MINA, really.
>
> -pete
>
>
> --
> proyal@apache.org - http://fotap.org/~osi
>
>
>
>
>
>
Re: MINA FTP Server
Posted by peter royal <pr...@apache.org>.
On Aug 17, 2006, at 11:47 PM, Mark wrote:
> Basically, by the time the FTP Server is done, I will have to
> support text
> and binary over the same connection, and support sessions. Trying
> to figure
> out what is the best I/O class for this.
>
> ideas??
Sessions are the MINA IoSession.
For text and binary.. There's probably some sort of state-machine on
the server side? Both are just bytes to MINA, really.
-pete
--
proyal@apache.org - http://fotap.org/~osi
Re: MINA FTP Server
Posted by Mark <el...@gmail.com>.
Basically, by the time the FTP Server is done, I will have to support text
and binary over the same connection, and support sessions. Trying to figure
out what is the best I/O class for this.
ideas??
On 8/16/06, Niklas Therning <ni...@trillian.se> wrote:
>
> I have no idea how the FTP RETR command works but maybe
> o.a.m.filter.StreamWriteFilter would suit your needs?
>
> HTH
> /Niklas
>
> Mark wrote:
> > yeah, a week or so. Well, I am getting close to the end here. One
> > question...
> >
> > What is the easiest way to write a file back to the client for this FTP
> > Server? I have just about everything done for a basic FTP server
> > except the
> > RETR command.
> >
> > Thanks..
> >
> >
> >
> >
> > On 8/1/06, Mark <el...@gmail.com> wrote:
> >>
> >> I am thinking that I will base some of the protocol logic on what the
> >> FTP
> >> server in the incubator has done, but I am not sure how much code I
> will
> >> use, as it is very tightly coupled together.
> >>
> >> I hope to have something up and running in the next week or so.
> >>
> >>
> >> On 8/1/06, peter royal <pr...@apache.org> wrote:
> >> >
> >> > On Aug 1, 2006, at 8:42 AM, Mark wrote:
> >> > > I was looking into writing an FTP server using MINA. I found code
> >> > > in the
> >> > > Apache incubator that I was going to use, but not sure if there are
> >> > > any
> >> > > logistical issues that may come back to bite me if I use code in
> the
> >> > > incubator. The FTP Server code seems to have good command handling
> >> > > and all,
> >> > > so I think it makes sense to use something that has been tested.
> >> > > What are peoples thoughts on this?
> >> >
> >> > No logistical issues with working with the incubator code. It
> >> > originated at the ASF, so there's no license issues.. The FTPD
> >> > project was moved there in an attempt to build a community around it.
> >> > Having a FTPD built upon MINA would be most excellent!
> >> > -pete
> >> >
> >> >
> >> > --
> >> > proyal@apache.org - http://fotap.org/~osi <http://fotap.org/%7Eosi>
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >
>
>
> --
> Niklas Therning
> www.spamdrain.net
>
>
Re: MINA FTP Server
Posted by Niklas Therning <ni...@trillian.se>.
I have no idea how the FTP RETR command works but maybe
o.a.m.filter.StreamWriteFilter would suit your needs?
HTH
/Niklas
Mark wrote:
> yeah, a week or so. Well, I am getting close to the end here. One
> question...
>
> What is the easiest way to write a file back to the client for this FTP
> Server? I have just about everything done for a basic FTP server
> except the
> RETR command.
>
> Thanks..
>
>
>
>
> On 8/1/06, Mark <el...@gmail.com> wrote:
>>
>> I am thinking that I will base some of the protocol logic on what the
>> FTP
>> server in the incubator has done, but I am not sure how much code I will
>> use, as it is very tightly coupled together.
>>
>> I hope to have something up and running in the next week or so.
>>
>>
>> On 8/1/06, peter royal <pr...@apache.org> wrote:
>> >
>> > On Aug 1, 2006, at 8:42 AM, Mark wrote:
>> > > I was looking into writing an FTP server using MINA. I found code
>> > > in the
>> > > Apache incubator that I was going to use, but not sure if there are
>> > > any
>> > > logistical issues that may come back to bite me if I use code in the
>> > > incubator. The FTP Server code seems to have good command handling
>> > > and all,
>> > > so I think it makes sense to use something that has been tested.
>> > > What are peoples thoughts on this?
>> >
>> > No logistical issues with working with the incubator code. It
>> > originated at the ASF, so there's no license issues.. The FTPD
>> > project was moved there in an attempt to build a community around it.
>> > Having a FTPD built upon MINA would be most excellent!
>> > -pete
>> >
>> >
>> > --
>> > proyal@apache.org - http://fotap.org/~osi <http://fotap.org/%7Eosi>
>> >
>> >
>> >
>> >
>> >
>> >
>>
>
--
Niklas Therning
www.spamdrain.net
Re: MINA FTP Server
Posted by Mark <el...@gmail.com>.
yeah, a week or so. Well, I am getting close to the end here. One
question...
What is the easiest way to write a file back to the client for this FTP
Server? I have just about everything done for a basic FTP server except the
RETR command.
Thanks..
On 8/1/06, Mark <el...@gmail.com> wrote:
>
> I am thinking that I will base some of the protocol logic on what the FTP
> server in the incubator has done, but I am not sure how much code I will
> use, as it is very tightly coupled together.
>
> I hope to have something up and running in the next week or so.
>
>
> On 8/1/06, peter royal <pr...@apache.org> wrote:
> >
> > On Aug 1, 2006, at 8:42 AM, Mark wrote:
> > > I was looking into writing an FTP server using MINA. I found code
> > > in the
> > > Apache incubator that I was going to use, but not sure if there are
> > > any
> > > logistical issues that may come back to bite me if I use code in the
> > > incubator. The FTP Server code seems to have good command handling
> > > and all,
> > > so I think it makes sense to use something that has been tested.
> > > What are peoples thoughts on this?
> >
> > No logistical issues with working with the incubator code. It
> > originated at the ASF, so there's no license issues.. The FTPD
> > project was moved there in an attempt to build a community around it.
> > Having a FTPD built upon MINA would be most excellent!
> > -pete
> >
> >
> > --
> > proyal@apache.org - http://fotap.org/~osi <http://fotap.org/%7Eosi>
> >
> >
> >
> >
> >
> >
>
Re: MINA FTP Server
Posted by Mark <el...@gmail.com>.
I am thinking that I will base some of the protocol logic on what the FTP
server in the incubator has done, but I am not sure how much code I will
use, as it is very tightly coupled together.
I hope to have something up and running in the next week or so.
On 8/1/06, peter royal <pr...@apache.org> wrote:
>
> On Aug 1, 2006, at 8:42 AM, Mark wrote:
> > I was looking into writing an FTP server using MINA. I found code
> > in the
> > Apache incubator that I was going to use, but not sure if there are
> > any
> > logistical issues that may come back to bite me if I use code in the
> > incubator. The FTP Server code seems to have good command handling
> > and all,
> > so I think it makes sense to use something that has been tested.
> > What are peoples thoughts on this?
>
> No logistical issues with working with the incubator code. It
> originated at the ASF, so there's no license issues.. The FTPD
> project was moved there in an attempt to build a community around it.
> Having a FTPD built upon MINA would be most excellent!
> -pete
>
>
> --
> proyal@apache.org - http://fotap.org/~osi
>
>
>
>
>
>
Re: MINA FTP Server
Posted by peter royal <pr...@apache.org>.
On Aug 1, 2006, at 8:42 AM, Mark wrote:
> I was looking into writing an FTP server using MINA. I found code
> in the
> Apache incubator that I was going to use, but not sure if there are
> any
> logistical issues that may come back to bite me if I use code in the
> incubator. The FTP Server code seems to have good command handling
> and all,
> so I think it makes sense to use something that has been tested.
> What are peoples thoughts on this?
No logistical issues with working with the incubator code. It
originated at the ASF, so there's no license issues.. The FTPD
project was moved there in an attempt to build a community around it.
Having a FTPD built upon MINA would be most excellent!
-pete
--
proyal@apache.org - http://fotap.org/~osi