Push publishing and dropped client connections

[Nyk Cowham <ny...@demotix.com>]

Hello all,

Earlier this year I implemented an ftplet to implement a push publishing
system that would queue a message to a beanstalk message queue that is
watched by a number or file and web workers that move the uploaded files
via scp to our webserver and posts metadata to the website. It works great
except for one really big problem.

The problem is that FTP does not send any headers regarding file size or
any other information that will help show when the file has finished
uploading. In my ftplet I use the onUploadEnd() method to handle an upload
event and to send the message queue message. However, if the client
temporarily drops the connection before completing the upload this handler
is still called and the message gets sent. When the client reestablishes
the connection it appends to the file and the onAppendStart() and
onAppendEnd() methods are called.

I am using 1.06 release -

My questions:

   1. Is there any way for me to distinguish, in the onUploadEnd() handler,
   that the upload is not complete, but rather a dropped connection? I don't
   think there is.
   2. When an upload append (APPE) request is sent do the
   onUploadStart/End() handlers get called as well as the onAppendStart/End()
   methods?
   3. When an upload unique (STOU) request is sent do the
   onUploadStart/End() handlers get called as well as the
   onUploadUniqueStart/End() methods?
   4. Is there any way at all to know if an upload is complete or whether
   to expect or anticipate that an append event might be forthcoming?
   5. Has anyone faced the same challenge who might be able to give me a
   few tips on how you resolved it?

Thank you for any help and/or guidance you might be able to provide. In all
Apache FTP has been awesome in implementing our push publishing. I think we
just might be hitting against the limitations of the FTP protocols
themselves, rather than anything intrinsic to Apache Mina.

-- 
Nicholas Cowham
Software Engineer
http://www.demotix.com

Demotix Profile <http://www.demotix.com/users/ncowham/profile> |
Twitter<http://twitter.com/NykOfTime>